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PREFACE 



For more thm a decade the Survey Research Center of the 
University of Michigan and the D ivision of HeaJth Interview 
Statistic of the National Center for Health Statistics (NCHS) have 
had a continuous contractual arrangement for the investigation of 
response problems in reporting health information in sampling 
surveys^ 

The contract program, which started in the late I950'i shortly 
after the initiation of the Heaith Interview Surveys began with a 
series of validity studies in wliich iamples drawn from medical 
records were compared with data collected by interview* These 
studies were desired to identify patterns of response bias as a 
basis for developing procedures to iniprovc rcportirtg. These 
investigations of levels of underreporting and characteristics of 
response patterns were evaluated in terms of respondent status, 
the attitudes and behavior of the interviewer and the respondents 
and nature of the events being reported. Theie studies are 
discussed in the sections, "Behavior in Interviews/' and "Inter- 
viewer Performance Difference** of this publication. 

The more recent studies, which have developed Out of findings 
of the preceding research, involved eKperimental procedures 
designed to improve reporting* Investigated were such procedures 
as the use of verbal reinforcement of the respondent, probing as a 
method of improving memory and information retrieval, and 
varying the length of questions in an attempt to increase 
respondent participation in the intcmew. These studies are 
described in the sections **The Use of Verbal Reinforcement in 
Interviews and Its Data Acduracys*' **Memory and Information 
Retrieval in the Interview," and "Queitton Length and Reporting 
Behavior in the Interview" of this publication. 

All but one of the NCHS studies summarized in this repgrt 
have appeared as complete research reports in series 2 of Vital and 
Health Statistics, The study by Cannell and Fowler (1963)^ on 
validity of reporting visits to physicians was not published in the 
series* The second study of inlerviewer-respondent interactions by 
Marquis and Cannell^ was done in 1969 under a contract with the 
Department of Labor, Manpower Administration^ All others were 
contracted for by NCHS. In addition, two NCHS studies not 
conducted by the Survey Research Center are frequently referred 
to here because they had as their subject iome of the same 
problems of reportingi one in 1967 by MadoWj the Stanford 

Research Institute report published as Series 2-Numbef 23^ ; and 
the other in 1961 by E. Balamuthj/ et aL, the Heilth Insurance 
Plan study^ most recently published as Series 2-Number 7^, 

Because these studies have had considerable interest for 
methodologiits and for survey reseairchers more generally ^ it was 
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thought useful £o review them in a single volume so that the 
sequence of the major lines of inquiry could be followed. This 
report docs not include a review of literature nor does it 
attempt to integrate underlying theories. It does present the 
findings in such a way as to make apparent their consistencies or 
inconsistencies, and docs discuss some underlying hypotheses, This^ 
compilation also allows more emphasis to be placed on interpreta- 
tion Mid explanation than was possible in the individual presenta- 
tions. 

In the concluding sections of this report the findings of 
the several studies are synthesiied, a model of reporting is 
developed, and a description is offered of how the research 
performed at the Survey Research Center (SRC) has been applied 
to collection procedures used in the Health Interview Survey (HIS) 
to improve the quality of the collected data. 

Since these studies were completed, much additional method- 
ological %vork has been conducted by the Survey Research Center 
focusing on experimental procedures for improving the validity of 
reporUng, This newer research at times confirms findings in this 
report, provides further support for these hypotheses and, at 
times, runs counter to some of the conclusions. Some of thes# 
findings can be found in a forthcoming report, "Experiment in 
Interviewing Techniques," summarizing research conducted by 
SRC for the National Center for Health Services Research. 

The contractual relationship between the SRC and the HIS 
does not consist solely of a financial arrangement; much of the 
research is the cooperative work of the two organizations* The 
Bureau of the Census has also been an active participant in several 
of the studies, both in the planning and data collection phases. 

Charles Cannell 
Program Director, 
Survey Research Center, 
University of Michigan 
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A SUMMARY OF STUDIES 

OF INTERVIEWING METHODOLOGY 

Charles F» Cannell, Kent H/MarquiSj and Andre Laurent, 
Survey Research Center, Institute for Social Research The University of Michigan 

INTRODUCTION 



Survey intendewing as a technique of data 
collection has deveioped from early attempti to 
collect simple demographic information to the 
current more sophisticated inquiries concerning 
attitudes, motives, and a wide variety of factual 
information^ Despite the increasingly complex 
demands on the survey intemew, methodologies 
for question construction and intemewer be* 
havior have not changed a great deal. 

Much research on interview method has been 
directed to the general problems of underreport= 
ingi to inaccuracies in interaew data due to 
intendewer bias or response error, and to the 
problems of recall and information retrieval* The 
inadequacies of the interview method have been 
well documented and the need for improved 
techniques in data collection is readily apparent. 
However, little has been done toward perfecting 
the interview procedure as a method of data 
collection* 

One reason inteiviewing techniques have ad- 
vanced slowly may be that interviewing has no 
comprehensive theory to draw upon for cause- 
and-effect relationships* Ideas about effective 
questioning must be drawn from fragments of 
piychological theory or, more oftenj from folk- 
lore, experience^ and common sense* Before 
major advances can be made, it is necessary to 
learn more about what happens in the intemew 
situation and to develop some theories about the 
cause-and-effect sequences that occur. In several 
studies of experimcntiJ intemewing techniques 



and their effect on reporting behavior described 
in this publication, an attempt has been made to 
identify the elements of the intemew process 
that are potential sources for improving data 
collection* 

UNDERSTANDINQ 
THE INTiRVIlW PROCeSS 

Influanca of the Interviewer 

Early attempts to investigate inaccurate 
reporting in interview surveys were focused 
primarily on the interviewer. Results of these 
early studies suggested that the interviewer's 
attitudeSs expectations, background, and physi- 
cal characteristics introduced important sources 
of bias into the household interview. 

In a 1929 pioneer study of interviewer effect 
reported by Stuart Kicefi it found that 
interviewers who were prohibitionists were 
likely to ascribe the sad plight of destitute 
respondents to the excessive use of alcohol, 
while socialist interviewers attributed indigency 
of their respondents to generally bad economic 
conditions* 

The influence of interviewer expectations on 
the interview process was demonstrated in 1942 
by Stanton and Baker.® Respondents were 
shown 6 geometric desipis and were later asked 
in an interview to select from 12 designs the 6 
which they had seen prcviously» The inter- 



y viewers were given informatibn" about 

which designs were originally shown to respond- 

: enti, but were purposely told the wrong six 
designs. During the interview, the designs which 
■ the intervies^ers thought were the correct ones 
were identified more often than the designs 
briginally shown to the respondentip 

Another type of study demonstrated a less 
directj but still powerful inteiviewer effect. 
Katz^ found in 1 942 that interviewers from 
working-class backgrounds consistently obtained 
more radical opinionSj both social and political, 
from respondents than did interviewers from the 
middle class. Robinson and Rohde® conducted a 
study in 1946 on attitudes toward Jews in which 
interviewers in one group were Jewish in appear- 
ance and those in another group appeared to be 
non-Semitic. The Jewish-appearing interviewers 
obtained significantly fewer reports_ of anti- 
Semitic attitudes than did the interviewers who 
appeared to be non-Semitic. 

Rtsponie Error 

In a household interview, a respondent can be 
expected to provide information: (a) that per- 
tains to items about which he is knowledgeable; 
(b) that he can remember at the time of the 
interview; and (c) that he is willing to report to_ 
an interviewer. Underreporting or inaccuracies in 
reporting on the part of a respondent may result 
from lapses in any or all of these three cate- 
gorics. ■ ■■ ■ 

Myers^ published data in 1940 from the 1930 
decenni^ census that showed a suspicious 
pattern of reported ages ending in zero (SO, 40, 
So, etc.); In the 1930's Twila Neelyi o found 
that one out of every nine families receiving city^ 
relief failed to report this fact. Perry and 
Crossley published data in 1950 showing that 
a comparison of interviews with agency records 
produced significant differences on such items as 
voting md registration^ contributions to the 
Community Chesty age, and ownership of a 
library card* , 

VaHdity stud comparing data obtained 
from interviews with data obtained from objec- 
tive records show discrepancies between the two 
sources of irtformation for topics such as bank 
accounts - ? ; airplane trips - * v pediatric his- 



torylS; work history and: public 

hcaith.3.4.17.18 • : : : 

One way to interpret underreporting on the 
part of the respondent is to consider it a 
consequence of poor memory. The disuse 
theory, described by Thorndikel? in 1913 in 
accordance with the findings in 1885 of Ebbing- 
haus,2 0 suggests that events from the more 
distant past are more likely to be forgotten than 
are recent events* Thorndike assumed that the 
sheer passage of time brings about a vveakening 
of the memory trace. Similarly, one could derive 
from the Gestalt theory^ ^ a prediction of the 
high probability of the respondent to forget 
, events of low impact, particularly with the 
passage of time. 

There isj however J another theory of forget- 
ting. According to McGeoch,^^ who first (1932) 
explicity stated the basic ideas of: the modern 
interferences theory later (1961) expounded by 
Postman,^* forgetting does riot occur in an 
absolute sense. Information does not disappear 
from memory but may be more difficult to 
retrieve from storage because of competing 
associations or interferences. Only the accessi- 
bility of information declines, resulting in a 
*Messening probability of retrieval from the 
storehouse. "2 5 This would indicate that under- 
reporting is a problem of retrieval, and that 
reporting can be^ improved by manipulating 
conditions that facilitate the recall of informa- 
tion. ■■ • ■• 

Problams Qf Reoall and Information Rtlrt^ieval 

There are two critical stages for a respondent 
who is asked to report information fropt mem- 
ory. First j he has to search for and ratriwe the 
requested information from his memory; then 
he has to transmit this information to an 
interviewer. While performance may vary ac- 
cording to the level of the respondent's general 
motivation or dedication to the role, it is useful 
to think of recalling and reporting as two 
specific variables that can affect the accuracy of 
data. For example, uriderreporting may result 
from failure of recall or from failure of com- 
munication. An example of the latter case is the 
tendency of the respondent to withhold 
threateniing or embarrassing information. A 
fertile field for study is the type of underreport' 



irig that results from the failure of the cognitive 
processes in searching for and retrieving informa* 
tion from memory* 

The three major activitas of the interviewer 
arer(a) question asking, (b) probingi and (3) giv- 
ing feedback* If a question is not properly 
worded, the probability of obtainiKg accurate 
data is low. A question that is improperly 
wordedj inserted out of conteHt, or that conveys 
to the respondent the type of answer the inter- 
viewer wants can produce data that are biased. 
Probing refers to repetition^ or rephrasing of a 
question or the addition of a new question to 
obtain an adequate response when a previous 
response has not been adequate* The problem of 
introducing unwanted bias into the data through 
probing is solved by distinguishing between 
directive and nondirective probes. Interviewer 
feedback consists bf evaluative statements that 
the intemewer makes after the respondent 
answers a question, Theye statements may con= 
list of verbalizations indicating approval, atten- 
tion, or understanding, varying from a simple 
"Um-hmm," acknowledging the successful com= 
munication of an answer, to an elaborate rein- 
forcement of the respondent's behavior. 

Several classic experimental studies have 
demonstrated that simple positive verbal rein- 
forcement- can have marked effects on adult 
performance. Taffel^^ in 1955 gave his experi- 
mental subjects a pack of 3-inch-by-5-inch cards 
each containing a single verb jn the simple past 
tense as well as a list of six pronouns. He 
instructed each subject to form- a sentence from 
each card beginning with any of the pronouns 
and using the verb. In the first part of the 
session, during which the experimenter remained 
silent, the subject showed a preference for using 
each of the pronouns on the card. In the second 
part of the session the task remained the same 
for the subjects, but the experimenter said 
'*lJm-hmm" or "AH right" whenever the subjects 
used either the pronoun "I" or "we" in con- 
structing a sentence. Consequently, the rate of 
using *1'V or "wc*^ increased significantly duHng 
the second part of the session. 

Resei^ch by Greehspoon^S showed that ver- 
bal reinforcement after a respondent mentioned 
a plur^ noun in a free-association test increased 
the rate at which plural nouns were named. 



Another method of demonstrating the effects of 
verbal reinforcement involves the occurrence of 
certain kinds of behavior in a casual conversa* 
tion setting, Verplanck^^ in 1956 was apparent- 
ly the first to publish results from this type of 
study. However, subsequent research^ ^ indicates 
that the conditioning effect obtained was prob- 
ably due to several extraneous variables (pri- 
marily experimenter cheating, conscious or 
unconscious) in using the procedures or report- 
ing the data. More recently, Centers- - has 
successfully shown that with great care, one can - 
obtain an increase in the rate with which a 
person gives opinion statements in a conversa- 
tion setting if such statements are reinforced by 
another person. 

In 1962, Kanfer and McBrearty^ ^ interviewed 
32 female undergraduates about 4 predeter- 
mined topics. During the first part of the 
interview, in which the women were handed 
cards designating four topics and asked to talk 
about each, the experimenter remained silent. 
During the second part of the session the 
experimenter reinforced the respondent when- 
ever she talked about a predetermined two of 
the four possible topics. Reinforcement con- 
sisted of a pbsture of interest, including smiles, 
and the phrases **I see," "Um-hmm," and 
"Yes," During the second phase of the experi- 
ment the students spent more time talking about 
the. reinforced topics than about those that had 
not been reinforced. 



Biai I ntrodui^d Through Intirviewer F^baok 

The foregoing studies indicate that inter- 
viewer feedback may have important effects on 
the amount of; information reported, but they 
reveal very little about how different kinds of 
feedback procedures affect interview data* 

Hildrum and Brown^ ^ were the first investi- 
gators to show systematically, in a survey 
interview setting, that interviewer feedback can 
produce response bias/ Two groups of 10 
Harvard University students were telephoned 
and asked their opinions about Harvard Univeri- 
sity's philosophy of general education. One 
p"oup was reinforced by the investigators (who 
used the word "Good") each time a favorable 
comment was made, and the other group was 
reinforced after each unfavorable comment. 



; ^ Responses of the group reinforced for positive 
opinions were significantly more favorable to- 
Ward Harvard's philosophy of general education 
than those of the group receiving reinforcement 
for unfavorable comments. Interviewer feedback 
applied in this systematic way produced a major 
distortion in the overall attitude responses. 

In 1957, Nuthman^* asked two groups of 
college students a series of questions about 
themselves. In one group the experimenter said 
"Good" when the respondent answered a ques- 
tion in a way that indicated scir-acceptance. The 
other group was given no reinforcement. 
Respondents who received reinforcement for 
self-acceptance responses gave more answers of 
this kind than did the group that was not 
reinforced* 

A. W, Staats and his colleagues have done 
several studies in this general area. In a 1962 
study^^ the experimenter said **Good/' "Very 
good," or /*That*s fine" whenever the respond- 
ent scored in a positive direction on sociability 
. items* Another group was given the same inter- 
view but ho reinforcement. Staats et ah found 
that the group receiving the reinforcement 
scored significantly higher on the iociability 
scale than did the group not receiving reinforce- 
ment. Studies by SingerSS and Insko^ ^ followed 

STUDIES OF UNC 
OF HEALTH EVENTS IN THE 

This section summarizes some major findings 
of validity studies about the reporting of health 
events and health-related behavior in the 
household interview* It focuses primarily on 
underreportings since health events are more 
likely to be underreported than overreported. 
Estimates of the magnitude of bias in surveys 
and caJculations of correction indeKes for data 
analysis are not fticluded in this discussions since 
the studies show only underreporting bias, not 
net bias. 

The five major studies discussed here were 
conducted For the National Center for Health 
Statistics- Their focus was not on the inter- 
viewer, but on the characteristics of the 



these general kinds of experimental designs and 
obtained sirnilar results. 



Re^arch Needs 

One of the conclusions that can be drawn 
from this background information on the inter- 
view process is that research on the improve- 
ment oJT reporting can fruitfully be devoted to 
the cause-and-effect relationship between the 
occurrence of different kinds of behavior or 
patterns of behavior and the validity of data 
reported. The behavior that occurs during the 
interview situation includes not only that of the 
individual interviewer and respondent, but also 
that involved in the interaction between the 
two. Behavior may be motivated by controlled 
interviewer feedback, techniques designed to 
hcilitate recall, verbal reinforcement, and an 
effective interviewing instrument^ namely, the 
questionnaire's s .3 9 

Obtaining good information in an interview is 
not simply a matter of asking many questions. 
More must be learned about the basic principles 
of memory and retrieval in order to provide a 
better understanding of the way in which 
information is stored and to devise more effec- 
tive ways of retrieving that information/ 

ERREPORTINQ 
HOUSEHOLD INTERVIEW 

respondents and their reporting patterns. Partic- 
ular attention was also paid to the nature of the 
information being reported/ In five studies 
similar questionnaires and comparable interview- 
ing procedures were used, ^d the reports of 
respondents , were compared with independent 
records assumed to be valid. The studies are 
identified as follows: 

• HIP: a study of the Health Insurance Plan 
of Greater New York, in which interview 
reports were compared with medical 
records;* 

• SRG: Three studies^ 7,18 conducted by 
the Survey Research Center, in two of 



>vhich re^ of hospitalizations were 
conapared with hospital discharge 
records,*^'!® and a third in which reports 
of physician visits were compared with 

■ clinic records ;V 
\ • SRIi a study carried out by the Stanford 
Research Institute in which respondent 
reports were checked gainst physician 
records.*^*^ 



Since the studies were designed to investigate 
validity of reporting and were directed toward 
problems of underreporting, they were based on 
samples of records of presumed high accuracy. 
Hospital discharge records, clinic records, and 
physicians" records were used as sample frames* 
Samples were usually weighted for certain 
characteristics and in some cases certain types of 
records" were omitted from the sample. (For 
example, in the second SRC study of hospitali- 
zation reporting,- ® normal deliveries were 
omitted and the sample was weighted with 
hospitalizations of more than 3-months' dura- 
tion.) 

In each of these itudies, interviewers were 
^ven the family names and addresses of the 
respondents/ Usually a dumniy sample was also 
drawn from the phone book or city directory to 
help disguise the aims of the study* Interviewers 
were told that the study was special, but were . 
not told its purpose* Form d inquiry conducted 
after the studies were completed showed that in 
no case had an interviewer guessed the study's 
true purpose* 

Interviewers were either experienced in the 
Health Interview Survey of the National Health 
Survey, or were given a 2-week intensive training 
session. Standard intemewing techniques of the 
Health Interview Survey were used in these 
studies. While the qiiestionnaires differed in 
some waySy they were all essentially the same as 
those used in the Health Interview Survey. 

In four of the studies^ * * ^ * V® the usual 
procedure of using proxy respondents was 
followed. The interviewer personally questioned 
all adults who were home at the time and used a 
proky respondent for all adults not present and 



more recent report from this study is Vital and Health 
Statistics^ Stties 2, No. 57. 



for all children. One study (SRI)^ included only 
self-respondents. 

The analysis consisted of matching repdrts l 
from the interview with information contained 
in the medical records. The first part of the : 
analysis involved an examination of the 
relationship between the characteristics of the 
health events investigated and the patterns of 
underreporting. The second part of the analysis 
was confined to the relationship between 
characteristics of the respondents and patterns 
of underreporting. 

UNDERREPORTIISIG AND CHARACTERIS- 
TICS OF HEALTH EVENTS 



EffiQt of Elided Time on Reporting 

Investigatprs have long been , aw^e of the 
limited timespan : over which a person gives 
accurate reports* However, few studies have had 
adequate data to demonstrate the extent to 
which this phenomenon occurs. 

Table 1^ demonstrates the decrease in 
reporting of hospitalization that occurs as the 



Tible 1^ Numbir of ricorded hoipltal diichargas and perctnt - 
not fiported in Intirvliv^, by timt elapsid bttween diieharge ;_ 
and fntervlcw: iurviy Rgs#arch Centir 



Time elapied 


Ricorded 
dlseharges 


Pi rein t 
. not re- 
ported 


1^10 wtaks 


114 


- .-. ^ 


11*20wteks . , / 


426 


6 


ai»30wttks . 


459 


9 


Sl^Oweeki 


33S 


11 


41-50 weeks 


364 


16 


51-53 wtiki . , . . , , 


131 


42 



SourEei reference 17. 



^Moit tables tf€ baled on events (hdspitaUutlonsi vislti to 
physldans, chronie conditions}, not on penons^ Thm perion with 
two events thus has a wei^t of two. For hoipltalizmdonsi 90 
pcrcant are sin^e events^ and thus persons and events tend to be 
the same. For phydcian visita and chronic condidons, howeveri 
the concentimdon Is hl^er^ 

In tabl^ frora the same study, the number of events differs 
somewhat* For clarity of presentation some urelevant ^tegories / 
("not ascertained,'' for eicample) are omitted. Ihc full report of I 
these studies pves complete data^ 



interval increases between the date of the event 
: juid the dat€ of the interview. This appears to be 
a typical "forgetting" curve in which failure to 
report an event grows as time passes. The same 
curves are evident for both male and female 
V respondents. The curve rises more slowly for 
self-reports than for reports given by another 
family member. Both SRC studies of hospitaliza- 
tion* ^ ' showed very similar patterns* 

The HIF study^ also showed underreporting 
of hospitalizations (table 2), The numbers are 
not stable because of the small samplej but the 
mtes of underreporting and the general pattern 
are similar to those in the SRC studies.^ ^1-1 8 

Table 2. Number of recprded hospital admlisions and pgrsent 
not reported in interview, by time elapsed between admii' 
sion and interviews Health Insurance Plan of Greater New 
. York 



Time elapsed 


Recorded 
admissilons 


PerE^nr 

not 

porttq 


Lesf than 1^2 months * . . . . . \ / 


6i 


4 


2-imonthi , * * . * . . ...... 


82 


2 


i-S months ^ 


12 


4 


8^11 months * . * v * ... * 


7B 


20 


icm months 


42 


50 



Source: reference 4. 



The reader should consider that the intendew- 
ing took place oT/er a period of roughly 2 
months— from May 2 to July 6^ 1958* If the 
dates of hospital adrntasion are to be expressed 
as approximate irttervals from date of hospital 
admission to date of household interview, there 
are overlaps in the classeSj but rough equivdents 
are: ■ 



□ate of admission to hospitai 


Approxlfnate interval to 
household intendew 


lefere July 1957 * . ........... . * 


10-11 months 


July-September 19i7 


i*1 1 months 


OGtober-Deeember 19i7 .......... 


5-S monthi 


January<>March 19S8 . ^ , . . ..... . . , ; 


2«B months 


April^ne 1 958 . . . ......... . ... 


than 1^2 months 



In the SRC study on visits to physiciansjl 
respondents were asked to report visits over the 
2-week perioB preceding the week in which the 
intemew took place. As shown below* the rate 



of underreporting for the second week preceding 
the interview was twice that for the week 
immediately preceding the interview. 









Peroani 




Time elapsed 


Recorded 


noi^ re- 








ported 


1 week . . 




ifie 


IS 


2 weeks . 




202 


30 



In the SRI study on reporting of chrome 
conditions similar patterns of higher fates of 
underreporting occurred with an increase in time 
elapsed since the last clinic visit (table S)* 



Table 3. Nufnber of recorded chronic conditions end percent 
not reported in Intervieyvi, by tirm eleptad betMfeen 1^ 
etlntc visit and fntervlew: Stanford ReMrch Inivltyte 



Time elapsed 


Re^rded 
^nditlons 


Percent 
net re- 
ported 


1-7 days 


lis 


9 


8-14 days , . , 


218 




IMS days ......... . . . . . 


440 


24 


days . . * . . . . . . . ... . 


S83 


42 


§7-84 days . .... . , ... . , . . 


§74 


37 


ee^llldayi . . ... . . . .... . 


513 


42 


113*140divi ... . . .... . . . . 


478 


45 


I4l^ie8days . . . ...... . ... 


asi 


46 


1i9^224days . . . . . ........ 


372 


57 


2»280 dayi . . . . .... . .... 


1,232 


12 


281^364 days ... . . . . , . . . . . 


1,078 


BB 


365 days or ntiore ........... 


71 


m 



Bqutobi reference 3." 



Simile data for the reporting of dufonic 
conditions for both checklist recopution ques- 
tions and nortchecklistj free-response itemi from 
the HIE study are shown in table 4* ^ 

Although the phenomenon of increase in 
underreporting over time is evident for both 
hospitalizations and chronic conditionSj the 
shapes of tfie curves differ. The curve, for 
hospitalization underreporting increases slowly 
during the 6 months following the event, but 
increases sharply beyond that period* The curve 
for the underreporting of chronic conditions 
rises rapidly during the first few weeks after the 
visit to the clinic and then flattens out after a 



6 



17. 



Table 4,; Percent of recorded Ghronfc conditions, ^ checklist 
: itatui, which were not Tiportid in houithetd interviiWi; by 
time eiipied between last Rlintc viiit and interviiwi Hsealtb 
Insufunce Pfan of Gireater New York 





Feroent not rfperted 


Timtelapiid 


Conditlonj 
an chiakitit 
recognition 
lilt 


Contfttioni 
not on 
chickliit 
recognition 
list 


Less than 2 wf eks , . ^ . , . . 


32 


Si 


2 vvf eki-4 monthi , , . : . \ \ 


51 


79 


4 months or more . . , , 


m 


84 



Source i re ft re nee 4. 



few months. It is interesting to note that data 
from a feasibility study conducted in Chester, 
England; Smederevo, Yugoslaviai and Chitten- 
'denj Vt.^^ showed that there were significantly 
fewer visiti to physicians reported for the 
second week preceding the interview than for 
the first week (see table 5). Similar data on the 
underreporting of both medically attended and 
nonmedically attended illnesses over a 4-week 
reporting period were found in the California 
Health Survey (see table 6).4f 

Perhaps the bes*: documented phenomenon of 
underreporting of health events as well as of a 
wide variety of other types of events and 
behaviors, is the decrease in^ the reporting of 
events as time elapses. This is characteristic of 
studies of consumer purchases, reports of 
income, behavior of children as reported by 
parents, and so forth. Some investigators have 
hypothesized that this decrease in reporting is 
not a result of forgetting but is due to the 
tenden,cy of the respondent to misplace the 



Tibfe i. Perctnt diitpibutlon of reported physician visIti, by 
week of occurrence reported in interview. In three selecttd 
■ areas , , 



Reported 


Cheiter, 


SmederevO; 


Chfttenden, 


occurrence 


Engtind 


Yugejfavii 


Vermont {UM^ 


Last week . * . * 


57 


Percent diitril 
i3 


lUujsn 

■ :s7 


2 weeks ogo ... 


43 


47 


43 



Sourcei reference 40. 



Table l/ illneii/ii retried for i 4^aek reealf perK^ exprtsseU 
as a per^ntage of the number riported in the -^.^t Week «9f 
the rtcaif pedodi California Health Survey 







Hlfleises 


tUneisas 






r with 


• without 


Reported occurrence 


Total 


ic:tivlty ~ 


activity 


of iUness 


Ulnesiei 


\ fcitrilnts 


reiiralnts " 






or medical 


or medical 






ittendaftce 


attendance 




Percent riported 


Last week . . . , . . 


1QQ 


100 


100 


2 weeks ago , . ^ , , 




m 


48 


3 weeks ago . , > . . 


40 


as 


as 


4 weeki af 0 ^ ^ . , . 


3S 


m 


; ■ 26 



iourcer reference 41. 



event in time and recall it as being outside the 
reference period. This Mplanatlon is especially 
relevant to the sharp increase in underreporting 
of events of the very last (earliest) weeks of the 
reporting period. While these studies do not 
provide a conclusive mswef, some of the 
findings strongly suggest that misf placement in 
time does not explain^a significatst amount of 
underreporting. 

In the SRC study of the reporting of 
physician visits^V respondents were asked to 
report visits made during the 2 weeks preceding 
the week of the interview. However^ the sample 
was drawn to include persons who had had visits 
within 4 weeks of the interview. If the 
underreporting in this study were due to random 
misplacement of the event in timej one >wouldr 
expect compensatory overrepbrting of physician 
visits from the third and fourth week to be 
reported as having taken place in the first and 
second week preceding the interview:. This did 
not occur. Telescoping into a more recent time 
period accounted for only a small amount of the - 
reporting error. In one experimental study in 
which the usual 1 2-fnonth . reporting period for . 
hospitalizations was lengthened to 18 months, 
the data were compared to see whether known ' 
events were inaceurately reported as having 
occurred in the 12-18 months prior to the 
interview. This was not the case. v/ 

When respondents who did report their 
hospitalizations were asked for the month of 
discharge^ 82 percent correctly stated the month 
and only 3 percent were in error by rnore than 1 



: month ill dther idkection* For those who 
\ miiplaccd the moii'h of the hospitalizatibn there 
_ wm no predommaht pattern of repprting the 
event as having occurred eai'lier or later. 
Furthermore, respondents were as accurate in 
leporting the month of discharge for hospitaHza- 
tions that occurred between 45 and 52 weeks 
prior to the interview as they were in reporting 
those of the most recent weeks. For visits to 
physicianSj over three-quarters of the reported 
visits were accurately dated to within a day. 

Theie findings present strong evidence that 
the increase in underreporting as time elapses is 
I not primarily a function of the respondent's 
inability to place the event In time. One must 
look to other sources for an adequate 
explanation. 

Effiat of Impact of the Event Upon 
Roporting 

Since early studies of memoryj it has been 
recognized that the greater the impact of the 
event upon the persons the more readily it is 
recalled, ^Impact is a term that is poorly defined 
but generally refers to personal importance or 
significance of the event. Psychologically, ^ it 
suggests that tcertain events occupy a greater part 
bf one's psycMc life, having greater relevance 
than ether events for one-s present life* In this 
section some indexes of impact and their 
relation to underreporting are examined/ 

Both SRC studies of reporting of hospitaliza^ 
tions^^*** clearly demonstrate that the longer 
the duration of the hospitalization, the lower 
the rate of underreportingv Table 7 shows the 
results of one of these studies* 



According to the tWrd SRC study the level 
of underreporting of physici&a visit s was lower 
when two or more such wits had occurred 
within the 2 weeks prior to the interviewi 



Recorded Individual viski within 2 
Wttki prior to intdryiivy 


Toti^ 
rtcorded 


Pf rsint 
not rt- 
porttd 








. . , . . . . 


W7 


29 




/ 110 


21 


3 or more * , * * * * \ . 


S6 


13 



In the SRI study a similar decrease in the 
underreporting of chronic conditions wu noted 
as the number of clinic visits relating to the 
condition increased (table 8).* 

Table 9 demonstrates another inde^ of 
impact. Reporting of automobile accidents was 

Table 8. Number of re^r^d ehronle conditions and percant 
not fiporttd In Imirvlewv^, by ;numbf r of Individual visits to 
ellnlci Stanford Ras^rch instituts 



Individual visits 


Rf€ordad 
^ndltions 


Parmt 
not ra- 
ported ^ 


1 . , , , , V. 




6S 


2.\ 1 




47 


3. .... . . 


S43 


36 


4 ; V, . 


m 


26 


B or mora . . ^ , * ^ ^ / . , , . . 


496 


14 



ioufctr refertn^ 3. / ; ^ 

Table 9. Numbir of rtcordad sutomobna aeeldana, both 
involving pirsonal infury and net* and paroant not raported 
in Imarviaws, by tima alspfad batvmn a^Want and intafvlevv 



Table 7. Numbar of raeordad hospits! disghargas and parcant 




A^ldants with no 


Aecid^nts with 


not raported In Interviewf^ by raoordad duration of hospital^ 




per^nal Injury . 


personal Injury 


latlon (eicelyding ovarraoortsh Surs^v Research Center 


Time elapsed 












Fer^nt 
net re- 
ported 




Parent 
not fi' 
ported 


Raaofded duration 
. of hospitalization 


Rfcorded 
diichargei 


Percent 
not re- 
ported 




Recprdad 
numbers 


Rtoorded ; 
numNri 






Less than 
3 months , 










I day . . V . . .\ . . . . ...... 


150 


26 


4i 


6 


71 


1 


2<4dBys . ..... . ^ ..... . , 


646 


14 












6-7 days ^ ............. . 


456 


10 


months . . . 


68 


12 


141 


10 


S-14dsys / . ........... . V 


352 


10 












1S-21 days. , . . ........ . . 


111 


i 


months . * * 


48 


22 


71 


10^ 


2M0dtys. - * V- - ........ 


5S 


2 


9-12 










^ 31 dayi or more ........... 


46 


S 


monthi. ... 


49 


37 


94 


22 



Source: rrfarence 17. 



Sour^: rafaranca42. 




ERIC 



more complete, rega-dlesi of the mterval since 
the acddenti if personal injury waj involved**^ 
Other e\id€nce of the relEtionshlp between 
impact and reporting can be lummeui^ed briefly* 
HoipitaUEations that included surgical proce- 
dures were more completely reported thw those 
not involving surgery. Condi tioni are more likely 
to be reported if the respondent saya he has pain 
and discomfort i 11 limited in activityt takes 
medidnes or treatment, or is concerned about 
his health* 

Tables 10 (HIP) aid 11 (SRC) show the 
effects of both elapsed time and impact on 
reporting* The cell totals for the chronic 
conditions in table 10 are small arid the results 
show some instability, but the previously noted 
pattern can be observed. 

Table 10. Number of recofded wBfV'iai vlsltl pnd pin:tnt of 
noncheekllit chronie ^nditlons not ffiporttd In Intervlev^; 
by time elapsed between last visit and Interylewi Htilth 
I niurinct Plan of Greater New York ^ 





Regbrdi^ visits 


Time etapied _ 








10 or 


1 














more 




Percent €ondlt|ons not 






rep'iPtid 




Less than 2 weeks ... / , ^ , . 


70 


71 


25 


37 


2 Meks^ mQnths . .... «... 


S3 


7i 


SO 


67 


4 months ©r mere * ^ ^ . , . . . 


S9 


Si 




19 



Sourer refeftnee 4« 



Table 1 1 . Recorded duration of hospitalizations ^nd percent of 
diichargfi not rtpofttd In Interviiv^p time elspsed 
between dlichar^ $t\d Interview: Sumy Research €enter. 



Time elapsed 


Duration of 
hasp! ta Illations : 


1 

day 


2-4 
dayi 


S diyi 
or more 




Per^nt dlieharges not 






reported 


1«20 weeks ^ , 


21 


5 


5 


21^w^ks. . .... , . . . . . . . 


27 


11 


7 


41^§2 weeks. . ^ 


32 


34 


22 



Sour^: referent 17* 



These data suggest that impact level of the 
event is clearly related to Ae adequacy of 



report. Furthermore, Acre is gn interactive 
effect of impact and time elapsed between the / 
event and the intemew. Neither of tiKese 
relationshipi is new or surprising! they conform 
to earlier findings md to theory. What is 
iurprising is the rapidity with which the curve of 
underreporting riies, especially for chronic 
conditions, and the strong effect of impact on 
mediating the effects of time on reporting. 



Effeat of Social and Personal Threat 
Upon Reporting 

Another factor that affects accuracy of 
reporting is the level of threat or embarTassment 
that the requested information hoMs for the 
respondent. Much research by socid psycholo- " 
psts emphasizes the effectiveness of group 
norms in bringing about and maintaining 
approved behavior among group members. Alio^ 
one*! perceived selWm age tendi to censor 
commimications so that the image is maintained, • 
The; study of hospitaliEation* ^ hai some " 
findings on this issue. 

A "threat scale" wm created for the 
hpipitalization study. .The diagnostic cl^sifica- 
tion w^ a J-poiut scale which, in the judgment / 
of the rciearchers, described the threat or 
embarr^smmit involved with the diagnosii. All 
diagnostic classifications that, in the opinion of 
the raters, would be very embarrassing or 
threatening "were placed in rank 1. Rank 31 - 
included the groupi judged neither embOTassing 
nor threatening. R^k 2 contained a mixture of ~ 
categories that were thought to be somewhat 
threatening or that might be threatening to some 
persons but not to others. Thus ranks l and 3 
were kept as , pure as possible, and : rank 2- L 
contdned the uncertain categories. The results 
of thii threat scalcj shovm in table 12s indicate 



Table 1 2. Numbtr of re^rded hoipltal dliehargii and percent 
not reporttd In intervlevys* by dlagnostl0 threat rating: 
Survey Rtseafch Gentir - 



Diagnostic thrfat rittng 


Re^rded 
dlioharges 


Fir^nt 

not re* - 
por^d 


Very ihireatening - . * ^ , 


235 


21 


Somewhat threatening ^ . ^ ^ ^ . ^ . 


421 


14 


Not threisening * . * . . » . , * , , , 


1,164 


10 



5our«!2 referent 1?. 



Tabft 13« Number of r^orded HospitslfzatiQns and percent not reported in interviews, by length of itav, time elapsed between 
- . , dUcharif and mtii^lfw^ end diignostic threat fating: Survey Rts 



Length of stay and time elapsed elnca diicharge 



Recorded 
hoipltalh 
. lations ^ 



Diagnostic threat rating 



Most 
threatening 



^nrjewhat 
threatening 



Least 
threatening 



Stay of 1»4 days 

Discharged: 

1-20 weeks ago * » * . . . , 

21-40 weeks ago > , 

41-i3 weeks ago . , . . 

Stay of S diys or more 

Discharged' 

1'20 weeks ago , . , 

21-40 weeks ago . ,\ . . . * . . * 

41-B3 weeks ago . . . * » » * . 

:. "Source: reference 17* 



223 
35g 
219 



308 
442 
273 



Percent not reported 



7 

26 
66 



0 
15 

33 



9 
1€ 
27 



7 

S 
22 



7 
9 
27 



3 
5 
17 



^ that highly threatening or embarrassing informa- 
tion is reported significantly less often than is 
nonthreatening information. 

From table 13* which shows a three-way 
effect of threat, impacts and time elapsed since 
hospitaUzationj it can be seen that there is a 
low'level relationship between the threat level 
and completeness of the case of most recent 
events* For less recent hospitalizations the three 
factors combine to produce mM"ked differences. 

By matching diagnoses from SRC interviews 
with hospital records two sources of reporting 
error were found: (a) complete failure to report 
the hospitalization; and (b) reporting the 
hospitali^^ations but misreporting the diagnoses. 
A few diagnostic categories*^ showing extreme 
differences between interview data and medical 
records are examined in table 14. As predicted, 
those with the lowest reporting levels contain a 
high proportion of probably threatening diag- 
noses. There are, of course, reasons other than 
embarrassment for not reporting a diagnosis 
accurately; for example, the respondent may not 
know the diagnosis, Howeverj it is likely that the 
differences between the two groups are due to 
differences in threat rather than to other factors. 

Underpaduates at The University of Michigan 
were asked about their hypothetical willingness 
to report each of a group of diagnostic 



Table 14. Number of diagnoses reported In Interviews and per^ 
cent of reported diagnoses compared with hospital records, 
by selected grouped diegnosefr Survey RMearch Center 







Peroent 






reported 


Diagnoitio groups 


Re€orded 


eompared 


diagnoses 


with 






hospital 






records 


Benign end un^clf led neoplasms . . 


87 


+51 


Infectious end per^ltic diseasei .... 


23 


+45 


Ulcer of stomach and duodenum 


38 


+12 


Diseases of the gall bladder . . . . . ^ . 


48 


+10 


Other digestive system conditions r . 


72 


-37 


Ptmale breast and genital disorders . 


S2 


—44 


Diseases of nervous system ynd sense 






organs ........... . ^ . , . ^ . , 


47 


-48 


Mental and pei^onallty disorders . . . 


8 


-67 . 



''coded according to Manual of th& Intarnational Smt^ical 
Cfamif/cstion ef Diseas&s, Injurm^ and Causm Qf D^fA, 18BS 
revision {World Health Or^nizitlon, 1987). 

Soy rce'i reference 17. = 



conditionsJfi In table 15 these data arc 
compared with what respondents actually 
reported . in the HIP study. The diagnostic 
categories that the students were most willing to 
report were surprisingly. similar to those actually 
reported best in the HIP study. 



10 



21 



respondenti' reporting behavidr. Survey reports 
wee easily susceptible to serious biases in the 
reporting of health events, and differential 
reporting bias can result in misleading conclu- 
sions. By understanding the problems involved 
in underreporting and distortion j one can design : 
studies to . improve reporting in the interview 
survey* ; 



UNDERREPORTINQ AND CHARACTERISE 
TICS OF RESPONDENTS 

In the remainder of this section some 
relationships between reporting and reipondent 
characteristics are analyzed* Are particular 
respondents most likely ; to underreport health r 
events? If poor reporting is characteristic of 
some res^ndent groups, then the reasons for 
difTerential reporting can be examined and - 
experiments can be designed to discover ways to 
improve reporting. The variables selected for 
study were those found to differentiate attitudes 
and behaviors in other studies and which mighti 
therefore, be expected to show differences in 
the reporting of health events. 

Age of R^pandant 

Data in table 16 suggest that there is an age 
effect in the reporting of hospitalizations i 



Table 16. Numbir of Morded hospltilliatlDn^ Inaludlng end eiceludins dtllverl^, and psrcint not rippnid In intervlewi, by and 

type of i^ipondent; iun^ty RiSfflrch Canttr 



Age of rtspondint 




; /_ Type of re^ondent 


Re€orded 
h^pltilU 
zitfons 


Atl ; 

rtfpond^ 
ants 


Proxy 
dren^ 


Proxy 
adults 


Self- 
respond- / 




Atl hospital izat Ions 


Pepsint not reported in Interview 






792 


8 


16 


13 




Vfiri ..... 




esi 


10 


i 


11 


11 


S5 yea^ and ever 




3S0 


15 




22 


10 




Hospltalliations exgluding denverlts 












1M4yeart . . , 




487 


12 


16 


16 


6 


3&54 yean . . i . . 




631 


11 


S 


11 


12 


ii years and over 




349 


li 




22 


10 



^ Defined by rtlationship to head of the households not by age. 
Source: reference 17. 



22 ; : 

11 



: Table IS. Hypothetieal willmgness of studenti to report Mrtaln 
J [ j : mrtieal eonditlans and ptrcent of actual (ntervlew rtporti of 

^ , ' \ these eondltions, by medical condition 







Percent 




Per«nt 


valid 


Condition 


vvifling 


re^rts in 


to report 


household 




{79 students) 


interview 






<H|P) 


More serious conditions^ 






Asthma . . . ... . . , , . . 


§4 


71 


Heart disease * ^ \ * . ^ . . ^ 


58 


60 


Hernia ... , , . . . . » . , 


55 


54 


^ Malignant neoplasm . . ^ . . 


31 


33 


Mental dlse^e . ^ . . ^ . . . 


19 


25 


Genitourinary dlseaie ^ , . . 


14 


22 


Less serious eondltionii 






Sinusitis ........... 


89 


48 


Indigestion 


18 


41 


Hypertension . . . . '\ . . . . 


83 


4a 


Varicose veins , 


65 


42 


Hemorrhoids .... ^ ... . 


21 


31 



Source: reference 26^ 



Summiry 

Th€ data cited here present consistent 
patterns of repofting; there is a predictable and 
significant relationship between some character- 
istics of the information sought and the 
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: younger respondents^ underreport* 
ing. The apparently large differe reporting 
hoipitalizationi for children (defined by rela- 
tionship to the head of the homeholdj not by 
age) is not meaningful since the 35-'to-54-year 
age group reported so few, 

Self-respondenti tended to be predominantly 
female and those who have proKy respondents 
were predominantly male. Since the best 
reporting was for younger self-respondents, it 
was hypothesised that this superiority might 
have been due to the fact that hospitalizations 
of these respondents might have been heavily 
weighted with normal birth of babieSs a category 
almost perfectly reported* Deliveries accounted 
for nearly one-quarter of all the hospitalizations 
in this first SRC study** ^ The lower part of 
table 16 shows the underreporting exclusive of 
normal deliveries* 

The overall trend for increased underreporting 
of hospitalizations with age disappeared when 
deliveries were excluded* Self-respondents under 
35 still showed superior reporting and adults 
over 55 with proxy respondents showed less 
accurate reporting, A second study ^® of 
hospitalization revealed similar age patterns: 
younger self-respondents, showed less under- 
reporting and older persons with proxy respond- 
ents more underreporting. 

From the third SRC study,* a high rate of 
nonreporting of visits to physicians by respond- 
ents 55 years of age and over is shown beloWi 



Table 17. Pereent of recarded chronie condiiioni not repsrttd in 
Intirvltvvs, by agfe ind mn of reipondinti Stanford Refearch 
Institute. 



; Age Of ftspondant 


Recorded 


Percint 
not re* 
ported 


18*34 years * . . . . ^ . . . . . . 


121 


20 


35-54 ytirs * , * ^ . 


200 


20 


5&^74 yiari . / . \\ ^ * % ^ * . . * , 


79 


34 



In the SRI study ,^ in which all persons 
■ reported for them^lveSj chronic conditions were 
reported more accurately by older respondents 
(65 years and over) than by younger respond- 
ents- This is true for both mde and female 
respondents (see table 17)* A second study of 
the reporting of chronic conditions (HIP) 
confirms this pattern*^ 

These coritrasts in resporise patterns suggest 
that the problem of underreporting is not one of 
. memory which usually deteriorates with age. 



of responoent 


Both 


Male 


Female 


siKei 
















Ptrcent not repafted 


Totaf, ail ages 




44 


46 


1 7-24 years .\ , 


48 


35 


57 


25^4 years 


46 


.40 


46 


35^44 years / 


48 


48 


48 


45-54 yean ^ ............... ^ » . 


45 


40 


49 


SS-64 ytars . . , 


48 


4i 


47 


65=74 years * . 


3i 


40 


31 


75-Si years 


37 


41 


31 



ioureet referenoeS. ~r 

The seriousness or impact of the conditionj the 
number of conditions, or the frequency of 
physician visits may influence the reporting 
; leveL " '.y;■ .,.-.^ 

The SRI study demonstrated that the number 
of visits made to physicians during the yew was 
highly correlated with the probability that a 
known chronic condition would be reported*^ 
The natu u of the task is another factor which 
may explain some of the demographic relatioh- 
ships with differential reporting of hospitaliza- 
tion and chronic conditions. For hospitaliEa- 
tionSj the respondent was asked whether or hot 
any member of the family had been in the 
hospital at any time during the past 12 months* 
For conditionSp the respondent w^ given a list 
and asked whether or not he had had any of the 
listed conditions at any time within the past 12 
months. Since the conditions were chronic, the 
probability is high that if a resporident had had 
any condition at any tinie during the psist year, ^ 
he would still have haS it on the day of the 
interview* Thus, wherfe' bqlh questions appear to 
ask for recall, the aver≥ v^ik 
actually much longer fw iiospitalizations than ; 
for chronic conditions, ^STbis may ixplaih why 
the data show^ a' dccreme m repbrtirig /of > 
hospitalizations over the years, but no similar 
cffefcf for the" reporting of y cohditidns, ; 

While these patterns also reflect the' effects of 
other variables, it seen^s clear that a respon dent's 
age in itself will be a predictor of whether or not 
health events will be reported. 
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Six of R^pondent 

It. has been iuggested by: some investigators 
chat illncis is perceived ai some sort of weaknesi 
and is more appropriate to the female than to 
the male role. Admitting to illness may threaten 
; a man's sclf»image^andj thereforei he may 
underreport illnesses. 

Maintenance of the family health is perceived 
as the role of wife and mother. It can be argued, 
thenj that if one is to use a single respondent to 
report about the family's health, the wife should 
be chosen. (This assumes that illness of other 
family members is not perceived by the womm 
as a failure in her role performance/ which 
wQuId lead to the prediction of greater 
underreporting on her part.) 

On the reporting of chronic conditions the 
SRI study^ showed that males failed to report 
percent of their own conditions, while 
fem^es failed to report 46 percentp Similarly^ in 
the HIP study ,^ which compared male and 
female respondents reporting for themselves or 
for spouse and children, the reporting difference 
never exceeded 2 percent* 

Male and female respondents showed almost 
no difference in their reporting of hospitaliza' 
tionSj reporting either about themselves or about 
other adults or children in the family. Any slight 
differences were in the direction opposite from 
that predicted. Similarly^ there w^ no differ^ 
ence between male and female reports of 
physician visits in either the SRC or the HIP 
study* In the HIP study, male and female 
respondents reported as accurately for proxy 
respondents as for themselves. 

One should not make too many generaliza- 
tions from these results* It must be remembered 
that the interviewer queried all adults who were 
at home when she called. A proxy respondent 
reported for those not at home* Since 
interviewers usually worked during the day^ only 
people at home during the day were likely to be 
interviewed* The usual persons at home were 
housewives, retired or unemployed menj or men 
who were at home because of illness* The strong 
possibility existed that these men would be 
better reporters of health events both for 
themselves and for others in the family than 
males who were not interviewed, that is, those " 
who were neither retired nor sick. 



iducatioh of Respondent 

Since it has been found in some research that 
persons with more years of education are better 
respondents, reporting patterns were examined 
by education! stams. The first SRC study of 
hospitali^-ation showed an interesting patterni 
the best reporters were high school and college 
graduates (tabkv 18). The*second hosp^ study 
showed the same pattern for high school 
graduates but the sample size was too small to 
aUow separate considemtion of college grad- 
uates* Respondents who attended college but 
did not graduate were poorer reporters than 
were those in lower educational groups. Whether 
this pattern is meaningful or is a chance 
phenomenon is unknown* One could hypothe- 
sise that persons who were diligent enough to 
complete luccessfully their college educations 
may also be more diligent in fulfilling demands 
of other taskSj md thus would be better 
respondents. Neither study shows a particularly 
strong tendency for higher educated respondents 
to report more accurately than respondents with 
less education. 



Table 18. Numbtr of rtcprded hoipitalliitloni and perEent not 
rtportid in "intertfliWi, by educatipn of A'^pondinti Survty 
Risiifch Center 





Record^ 


Percent 


Edusatlon Qf respondent 


hoipltall> 


not re^ 




zations 


ported 


Less than high school graduate ...... 


829 


13 


High ichoQi graduate .............. 


646 


■ 7 


Some eollege . ^ 


180 


16 


College graduate ................. 


lis 


5 



iourcei referanct 17, 



In the SRC study of reporting physician 
visit the liigh school graduate group did not 
show the same pattern. As seen in table 19^ the 
eollege group showed much lower undeneport* 
ing than did other groups, but thp sample was 
not large enough to warrant any conclusion. 

Table 20 shows the underreporting of chronic 
conditions in the SRI study* by educational 
group* In this study education was reported for 
the head of the household rather than for the 
respondent. In this table the pattern of superior 
reporting with increased education is hot 
apparent. There is no indication here of less 



Tebia 19. Number of recorded phyiician visits and per^nt not 
: reported in interviews, by education of respsndenti Survey 
- Researgh Center 



iduaati@n of fifpondent 


Recor^d 
visits 


Ptreent . 
not rt- 
portid 


frSyeirs V 


121 


26 


1^3 yean high S€hosl 


132 


22 


4 yean high sahool ... ..... . . 


113 


23 


1 year college or rno re . . V . , . 


33 


9 



Source: referenoe 17. 



Table 20. Number of recorded chronic conditions and persent 
not reported in lnt@rvlevyi, by education of heid of houie^ 
hold: Stanford Research Initltute . 



Education of riipondent 


Recorded 
conditions 


Per^nt 
not re- 
ported 


Lesi than college » * * * * * . 


3,9S3 


43 


0 years * . 


114 


40 


1^ years ^ . . , . ^ ^ . . * . . . . 


IIP 


51 


M years . , . \ ^ ^ . ^ \ , \ , \ , . 


Mil 


\ 41 


9-12y€ars. . * , . ... . . . ^ , . , 


2,W8 


43 


College , . . . . , . . . . . , 


2,040 


BO 


1^2yeari ^ . . . . . . , 

3^ years * ....... . 

S years . ^ * . . \^ . . .... * * , . 


851 
640 
549 


47 

i3 
63 
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underreporting by high school and college 
graduates as was found in the hospitalization 
study* 

The one conclasion from these SRI data is 
that respondents with less than college-level 
education reported more of their chronic 
conditions than did those who attended college* 
In contrast, the HIP, study showed no consistent 
pattern of reporting by educational level, 

; Data from these studies point to no definite 
conclusion. One cannot generalize that respond- 
ents with more education are better at overall 
reporting than are those with less education. 
Why the patterns differ for the studies of 
hospitaliEations md doctor visits from those 
found in the reporting of chronic conditions is 
not apparent- ; L 



In 1965 Fowler** made an intensive analysis 
of reporting by educational groups in the Health 
Interview Surveys Based on systematic observa* 
tion of interviewer and respondent behavior he 
concluded that less highly educated respondents 
needed more help from the intendewer to 
perform adequately. They were less skilled at 
the respondent role. There was also the 
tendency for the Eess educated to have less 
information about the purpose of the survey and 
what was being sought in the interview. 
Interviewers tended to be more active in 
interviews with less educated respondents, 
helping them to perform more adequately. 
Fowler considers that the effect of education 
may be in the skill level it represents. However* 
why respondents would show peatcr skill in 
reporting hospitalizations than chronic condi^ 
tions is unclear. « ^ 

Family Inrame of R^pondant 

In other research it has often been foimd that 
family income level is a better predictor than 
either age or education, since income frequently 
reflects both these variables ^ well m additional 
motivational components. 

In the first SRC study, hospitalizations were 
better reported as faniily income increased (see 
table 21); in the second SRC study of 
hospitalization reporting* the same pattern was 
suggested. The reporting of visits to physicians 
showed no such trend; although the best 
reporters seemed to have annual family incomes 
of $10,000 or more* the sample size was too 
small to yield firm conclusions* 



Table 21^ Numter of re€erded hospitalizations and per^nt not 
. reported in jnterviev^, by annual fafnlly ineome: Survey 
Research Center 





Recorded 


Fer^nt ; 


Family Ineome 


hospitalU 


notre* 




sations 


ported 


Less than $2,D0Q - 


154 


18 


$2,00043,^ . . . . ... ... . . 


301 


13 


^,000^,999 ... . . , 


750 


10 


S7,ooo4s,eg9 , 


272 


g 


$10,000 or more ........... 


248 


8 



iour^: rffirtn^ 17. 



" V In the SRI study on the reporting of chronic 
coriditions, the best reporters were in the lowest 
income group, with no other pattern apparent 
^ (see table 22). The HIP study of chronic 
- condition reporting also -showed that persons in 
: familiei with annual incomes of less than $4,000 
were the best reporteri and here again, no other 
pattern was observablep As with educationi 
: differences in reporting by income groups are 
T not consistent among studies. 



Table 22. Numbtr of rtcordid shronlc odnditlans and percent 
not reported In mtirviews, by annual fimlly incomes 
Stanford Research Institute 



Family incorne 


Reeorded 
condition! 


Pereent 
not ri' 
ported 


Ltss than $3^000 . ,,,,,,,,,, 


638 


36 


$3,00fr$4,i99 ............ 


962 


46 


f5,^0-$i,99f . . , , , 


1,373 


4B 


iT^ooo-se^iss ..... 


1,516 


46 


ilO«OQO or more 


1p437 


47 
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Color of R^pondent 

The most consistent finding on characteristics 
of respondents, one which shows up in three out 
of the four studies, is that white respondents 
reported significantly better than those of other 
- races (see table 23)» This was true whether the 
respondent w^ repoiting for himself or for 



Table 23. Persent of reoordad hoipitstlzatlons, chronlo condh 
tlons^ and physldan viilti not report^ In Inttrvlewii by color 
of rtipondint ' / 



Coior of respondent 


Hospital izs' 
tlons 


Chronic 
condi' 
tioni 
SR|3 


Phyil- 
clin . 
vfilts 
SRC 
study 
34 


SRC 
study 

iV 


SRC 
study 

2^ 




Percent not reported 


White . 


10 


16 


46 


24 


Ail others . , * . 


16 


27 


SO 


22 



. Reference 17. '^Rtft rtnc© 3. 

^Rifirtnce IS* 4^gfg|,gnee 1. 
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Other family members* None of the studiw;^^^^^^^ 
involved enough non-white respondents to 
permit intra^oup arialysis. One can only ■ 
hypothesize about the reasons for the finding* 
On the surface the diftoences ^e too large to ^ v 
reflect educational or income factors; rather, 
they seem to reflect differencei in behavior by v^^^^^^^^ 
; -.color. . -_ ■ ;:v .:,.,V ■ : :;'y,;V ' -y'.- = 

A similar pattern did not appear in^ the SRC 
study of visits to physicians. However, since the ^ 
sample for that study came from participants in v i j 
a voluntajry health plan, respondents of colors - 
other than white who participated in that plan - ^ 
would be expected to differ in several respects 
from a random sample. 

There is no ready eKplanation for these : 5 
reporting differences* It may be that the white 
interviewer provokes luspicions in respondents 
of other colors. V It may be; part of the present yyj 
; cultural pattern for these respondents (espedaUy^:^^ p| 
Negroes) to be unwilling to divulge information. ' ? 
The ansv/ers await further experimentation. ^ " 

Rapbrting for Self VeRUS Reporting for 

Other Fimlly Mambir f 

In the Health Interview Survey each person at ; 
home v^hen the^ interviewer calls is interviewed J 
for himself, and a ^^responsible selected adult** ^^^i 
reports for persons not at home and for childreri ; 
under 17* As one might expect, the coniplete- ;^^ 
ness of reporting depends ^out whom the 
respondent is talking. It is tempting, in tenns of ; ; 
time and cost, ; to ;use proxy respondents/^^^^^ ;: 
However, the data suggest that the practice has 
some real dangers in terms of quality of 

responses. " ^ . .: ... . ' .. . "r :^^^^^^^^ 

Table' 24, covering results of the first SRC 
study of hospitdization reporting, shovvi clearly --^^ 
that the more distant the relationship of the t 
respondent to the person about whom informa- 
tion was being reported, the poorer the 
reportingp The incre^e in underreporting about ^ 
chdldren as compared with "self * or "spouse" "^^^1 
m^ be due to the natiire of children's 
hospitalizations, which are generally shorter and 
involve less serious conditions than those of . 
adults. Data in table 25 from the HIP study on 
reporting of chronic conditions show a similar 
pattern of reporting for children. 



Table 24. Number of raeorded hdspltilfzationi and percent not 
rfpQrted In intirvigy^, by relationship of riipondent to 
lamplf person: Survty Research Ctntir 



Reipondant relationihip 
to sam pit person 


Recorded 
hospital 
zatloni 


Ptrcent 
not re- 
ported 


Sflf-Tiipondint 


1,092 


7 


Spouse . , , . ^ , , , 


27B 


10 


Parent 


386 


14 


Other relation 


78 


22 



Table 2i. Nymber of recorded physician visits and perctnt not 
reported in interviews, by reiattonship of respondent to 
sample pirson: Survey Research Center 



Respondent relationship 


Recorded 


Parcint 


to sample person 


Visits 


not re^ 






ported 


Stlf-reipondent 


204 


25 


Parent 


103 


18 


Other relation 


98 


2B 



iour^i reference 17< 



SQurce: reference 17. 



Table 25. Percent of recorded conditions, by checklist status, 
which were not reported In lnterviev>^, fay relmionship of 
respondent to sample persons Health Insurance Plan of 
Greater New York 











Perce 


nt not 




reportid 


Respondent relationship to 


Condi- 


Condi- ' 


sample person 


tions 


tions 




on 


not on 




check- 
: list 


check' 
list 


Mf-respondent ^ ^ ,,,, . 


57 


79 


ipouse 


62 


79 


Nrent * , ... ^ , ... 


72 


82 


Other relation 


68 


72 









Source: reference 4. 



Reporting of visits to physicians^ was better 
for children than for self-respondentSs and was 
about the same for self-respondents and adults 
with proxy respondents (table 26)* The recall 
period in this study was only 2 weeks long^ and 
an adult usually accompanied a child to the 
office; these factors may have accounted for the 
relatively good reporting of children's visits,^ 



' mcthodoiopcml Investlptlon of the impact of the use of 

proxy mpondcnts In die Hedth Intern Survey conducted 
aftor the completlDn of this report U presented In Kovir, G*, 
and Wrightj R* A„ "An EKpcriment With AlternaU Reipo 
Rules in'the National Health Intend^ 

Statiitlcs Section* Proesedings of ihe Ammcan Statiitieal 
Assocktwn, pp, BlhSl6, M* G,, and Wilson, R. W. 

■ •'Perceived HcalUi Statui^How Good Is Proxy Rcponinf," 1976 
Social Statistic SeQ^ont ^ocesdihgs of the Ammcan Statisticai 
Assockthn,VQUUfpp^i9&^B0Qj 



Conclusions 

One cannot leave these findings on reporting 
characteristics of respondents without attempt- 
ing some explanations* The general picture that 
emerges from these data is that characteristics of 
the respondent are not nearly as consistent, nor 
as strong in their influence on underreportings as 
are characteristics of th^ event. 

One finds effects of age, education, and 
income which are not strong, but which are 
consistent in the reporting of hospitalizations. 
The patterns also tend to be consistent for the 
reporting of chronic conditions. The striking and 
puzzling fact is the divergent nature of the 
patterns— persons with higher education, higher 
income, and of lower ages are better reporters of 
hospitalizations and poorer reporters of chronic 
^conditions—Aithough^mMLy^ofnhr^iff^^ 
are not significant when viewed in isolation, the 
total impression is that the differences are 
meaningful and cannot merely be attributed to 
random error. 

It is likely that these patterns reflect the 
effects of other variables, as has been hypothe- 
sized here. The Langing, Ginsberg, and Bratten 
study^2 of underreporting of ca^^ lOans from 
loan companies shows a marked it vume effect, 
with higher income respondents being poor 
reporters of their loans. This finding can be 
understood in terms of soclJ acceptability. 
Higher income people probably perceive making 
loans at smdl loan offices as contrary to the 
norms of their group, Weiss^^ found that 
mothers in lower socioeconnrn^- groups are 
more likely to report that their;^ .lildren were 
forced to repeat a grade in school than are 
mothers in higher socioeconomic groups* Again 
the report may be made to be cbhsistent with 
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behavior perceived as acceptable. Another 
explanation may be that lower iocioeconomic 
^oups have more sicknesi; therefore, it has 
^eater impact and is reported better. Hospitali- 
zations, on the other hand, tend to be single 
events and thus may be more difficult to recall. 
That the task requirements are different in terms 
of rec^l motivation level are other tenable 
hypotheies. Research is needed to explain these 
phenomena. 

In the studies presented here there is no 
indication that special groups are characteristi- 



cally poor reporters, with the exception of 
persons of races other than white who are 
sufficiently consistent in showing high under- 
reporting to suggest that special resewch be 
devoted to them. 

The general con elusion from these studies is 
that research on iinproving reporting can most 
fruitfully be devoted to the nature of events and 
the factors underlying the characteristics of 
events. Problems of elapsed time, impact, and 
threat or embarrassment appear to be the most 
significant issues for research. 



BEHAVIOR IN INTERVIEWS 



Before effective theories about the cauie-and- 
effect sequences in the interview situation can 
be developed, there must be accurate descrip- 
tions and classifications of the material repofted 
in interviews. It is to help meet this need that 
the Survey Research Center has continued 
studies which describe the basic nature of the 
verbal interaction between interviewer and 
respondent. Since both SRC observation studies 
discussed here are available in full report form, 
this discussion will eliminate many of the 
methodological details and concciitrate on the 
major findings and their possible inipHcations. 



c. The interviewer's rating of the respondent 
following each intendew; 

d. A reinterview with the respondent con- 
ducted by a second interviewer within 2 
days following the oripnal health inter- 
view; and 

e. A staff interview with each health inter- 
viewer following the completion of her 
assignment. 

Complete data are available for 412 respond- 
ents from, a cross section sample of the area east 
of the Mississippi (excluding the extreme 
Northeast). About four=fifths of the respondents 



HEALTH INTIRVIEW SURVEY 
OBSERVATION STUDY 

The first SRC observation study*^ by 
Cannellj Fowler, and Marquis wa^ carried out in 
cooperation with the National Center for Health 
Statistics*^ Five kinds of measurements were 
taken for each respondent* 

a. Information about respondent demo- 
graphic characteristics and family health in 
the regular health interview; 

b*; A detailed account of the interviewer and 
respondent behavior as recorded by a third 
person observing the interview; 



report of this study may be obtained from the NationaJ 
Center for Heilth Statistics/ Fils/ and Health StatisticSf PHS 
Pub* No* 1000-Scries g-No. 26. ^ 



were women^ and about half of the respondents 
had less than a high school education* 
Experienced femde interviewers employed by 
the U.S. Bureau of the Census conducted the 
health interviewing* Another group of women* 
also employed by the U,S, Bureau of the CensuSj 
carried out the behavior observation. The 
reinterview with respondents was conducted by 
a Survey Research Center intervdewer, rather 
than the original he Jth interviewer, The staff 
interview with the health interviewer was also 
conducted by a Survey Research Center 
interviewer. 

Health Intervitw Survey Data 

The information that the respondent fur- 
nished about his own health during the regular 
health interview was used in this study to create 
a dependent variable. The dependent variable 
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was the number of chronic and acute conditions 
the reipondent feported for himself, with 
adjustment for gross differences in actual 
sickness which could be precUcted from knowing 
the reipondent*s age. The previously cited fuU 
report of the study details the rationale for the 
choice of this particuto dependent variable* 
Evidence is presented which su^ests that the 
number of chronic and acute conditions the 
respondent reports for himself is an indication 
- of the accuracy of other health data reported by 
him. 

Observation 

During the health interview an observer 
recorded what the interviewer Bnd respondent 
said and did. A wide range of behavior classified 
in small segments of easily identifiable acts was 
recorded for both interviewer and respondent. 
In order to record different kinds of behaviors 
the interview was divided into segmentSs each 
containing a specific set of questions. For each 
segment several particular kinds of behavior 
were observed and recorded. In this way, a wide 
variety of behavior could be recorded while the 
task was kept within the observer's capabilities. 

While the interviewer was still at the door, the 
observer recorded such things as: the time of 
day, how long the interviewer had to wait for 
the respondent to open the door, what the 
"intemewer saicl as she ihtroduce^"Kerierf and^ 
the study, how many questions the respondent 
asked, and who took what kind of initiative to 
get the interview started. The observer also made 
two ratings about how receptive the respondent 
had been to this point in the interview. After the 
actual interviewing started, the observer re- 
corded the occurrence of different kinds of 
behavior at different points in the intemew* 
Special attention was paid to irrelevant behavior 
which departed from the task of asking and 
answering the questions on the questionnaire. 
Among the <ategories used to classify this 
irrelevant behavior werei talking about the other 
person (such as giving praise), asking irrelevant 
questions^ and giving suggestions. Conversation 
about the respondent or his family, friends, etc., 
was also considered irrelevant when it was not 
directed to the specific question asked. Another 
major category of irrelevant behavior was 



humor, consisting of laughter, jokes, and other 
means of relieving tension* The observer also 
recorded the reaction which the other person 
had to each instance of iwelevant behavior. 
Reactions were rated on a 3-point scale, from 
"very encouraging" to **very discouraging*" 
Throughout the interview the obieiver kept 
track of the kinds of potential distractions 
present (children, other adults, TVj radio). 

During three separate parts of the intemew 
the observer concentrated on the question- 
answer interaction between the interviewer and 
the respondent* Seven types of behavior were 
recorded for the respondentr 

a. Adequacy of answer; 

b. Elaborateness of response; 

c. Inadequacy of answeri 

d* Need for clarification or repetitioni 

e. Checking with another person or with 
records; 

f. Reference to calendar; and 

g. Doubt about the adequacy of an answer* 

Five specific kinds of interviewer behavior 
were also counted. They were ^ 

a. Repeating the answer from the question- 
naire; 

b* Asking a question^ not on the question- 
— — naire,—which-did— not— suggest— an-answer-- 
(nondirective probe); 

c. Asking a question > not from schedule, 
which might have suggested a specific 
answer, or asking respondent if she agreed 
with a specific answer (directive probe); 

d. Ctoifying the meaning of the question; and 

e. Suggesting that records, calendar, or other 
people be coniulted. 

Several other attempts were made to examine 
different aipects of t^k-oricnted behavior. In 
one section the interviewer counted the number 
of times the respondent paused before pving an 
answer, the number of times the respondent 
asked for clarification or elaborated on m 
answer, and the number of times the interviewer 
asked addition^ questions* During one particu- 
larly difficult part of the interview, ipecial 
attention was given to the frequency with which 
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the respondent had to ask for helpi to the 
interviewer*! behavior when the respondent 
made such a request, and to the effort made by 
the respondent during this difficilt part of the 
interview. Between sections of the interview 
where the observer recorded task-relevant 
behavior, she recorded her impressions of the 
respondent's reactions^ For examplej she rated 
the respondent's attitude (enthusiastic, bored, 
irritated), his understanding of the question, and 
the smoothness of the interaction between 
interviewer and respondent. 

At the end of the interview the observer 
fecorded the length of time spent in conversa- 
tion after the last question was asked and tried 
to determine whether the interviewer or the 
respondent waj more willing to continue this 
conversation. After the interview was com- 
pleted, the observer filled out two pages of 
ratings on the respondent and recorded her own 
impressions of the interview, 

Intarviiwer Ritings of the Respondent 

After the health interviews the interviewer 
rated the respondent by describing her own 
perceptions of respondent attitude and her own 
attitude toward the respondent. The rating 
scales used by the interviewer were similar to the 
^ones_use_ d_b y the obser veT at the end of each 
health interview, 

Riinterviaw With the Roipondent 

A major attempt was made to ascertain the 
respondent-s reactions to being interviewed by 
conducting a second interview within 2 days 
following the original health interview. The 
questionnaire used in this reinterview focused on 
the respondent's feeHngs and attitudes about the 
interview and interviewerr his level of informa- 
tion about surveys in general and this one in 
particular, his motives for cooperating with the 
intemewer, and his feelings about the questions 
and about his role as a respondent. A special 
attempt was made, using semiprojective tech- 
niques, to uncover any negative feelings that the 
respondent had about the original interview that 
might be difficult to express directly to another 
interviewer. 



Interview With the Htalth I mtrvlewer 

After all her obierved interviews were 
completed, each of the 35 health intemewers 
was in turn intervnewed by an SRC interviewer. 
The health interviewer was questioned about her 
attitudes toward herjobj her feelings concerning 
the interviewing of different kinds of respond- 
ents, her reactions to specific aspects of her 
work, and her reactions to the questions on the 
health interview scheduICp 

Results 

It was originally hypothesized that several 
kinds of respondent psychological variables 
would have a major effect on the quality of data 
reported. Specifically, it was felt that reporting 
accuracy would depend on the amount of 
information which the respondent had about the 
interview and its sponsors in combination with 
the fespondent's general attitudes, motivation 
patterns, and particular perceptions of the 
interview. It was expected that the information 
level, attitude, motivation^ and perception 
characteristics of the respondent would also be 
reflected in the behavior observed in the original 
interview. 

This attitude-based interpretation of the 
causes of accurate and inaccurate reporting is 
not new* Experience has been accumulated over 

many years (both from the pSVchological 

laboratory and from the world of advertising 
and marketing) which would enable the 
researcher to design techniques to change 
respondent attitudes, motivations, and percep- 
tions and to supply information or correct 
misinformation. Based on the assumption that 
p oor reporting was due to such variables as low 
tevels of information and inappropriate atti- 
tudes, new studies were designed and testing was 
started on some attitude- and information-leveU 
change techniques. However, as the main data 
became available, it became clear that the 
hypotheses concerning the causes of poor 
reporting were unsupported. There was practi» 
csdly no correlation between the dependent 
variable (a measure of reporting quality) and the 
complex indexes of information level, attitudes, 
motivation, and perception. (See Mueller, 
Schuesslerj and Costne^^ for further informa- 
tion.) These cognitive variables were for the 



most part also unrelated to behavior* Some of 
these data are reproduced in table 27. 

As more and more of the data were analyzedj 
it became apparent that the actual behavior in 



Table 27. Relationship of rtipondfnt cognitive \^rlablei to 
reporting indeic and behsvlor \nd§xB% 
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the interview the main variable that 

corralated with the index of reporting quality. 
Thus it appeared that if changes were to be 
made in the accuracy with which rcipondents 
furnished data about their health, behavior 
patterns in the conduct of the interview would 
have to be alteredi some kinds of behavior might 
be more conducive to good reporting than other 
kinds. To test thiSi correlationi were obtained 
between the reporting index and the frequency 
of various kinds of behavior which took place 
during the interview* The preliminary results 
suggeited that the kinds of behavior normcJIy 
considered task oriented (aiking for clarifica- 
tions ^ving elaborations upon aiiswers, and 
consulting records, on the part of the respond- 
ents arid probing by the interviewer) were more 
highly conrelated with the dependent variable 
than the kinds of behavior which are considered 
to be less relevant to task performmcej such as 
talking about self or joking. To illustrate, the 
relationihip between respondent behaidor and 
reporting^M 
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Beth c^ffielgnts are slpiifleantly different from zero (p < 



Further investigation revealeds however, that 
the frequency with which any one category of 
behavior occurred in the interview was highly 
correlated with the frequency wth which any 
other kind of behavior occurred. Thus it was not 
found that some intervdews were predominantly 
ti^k oriented and others predominantly inter^ 
personally oriented. What was found was a 
general beha^or activity level characterizing a 
particuto interview. The higher the beha\ior 
level (the more frequently each kind of behavior 
occurred for both interviewer and respondent), 
the higher the score on the reporting index. 

Since it was impossible to determine which of 
the behavior categories, if any, determined this 
general activity level, it appeared logical to 
ascertain who was responiible for setting the 
activity levels. Since the data are correlational, it 
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is difficult to determine directly whether the 
reipondent or the interviewer had major 
responsibility for determining the amount or 
level of Ichavior in the interview. However, it 
was determined that intervdewers themselves did 
not have a characteriitic behavior level for all 
interviews* The data also indicated that there 
was an extremely high correlation between the 
level of behavior of the interviewer and the level 
of behavior of the respondent- 
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Thus it appeared that the amount of behavior in 
the interview tended toward some sort of 
balance* If the interviewer engaged in a high 
level of behavior, so did the respondent, and vice 
versa. It w^ also noted that the balance was 
most likely to occur when the behavior levels of 
the inteiviewer and respondent were either 

, especially high or especially low, A special 
statistical treatment of the behavior index data 
which shows the high probability of balance at 

'~extreme^behavior^levelsHi'-given^ m-thc Driginah 
research report by Cannell, Fuvvler, and 
Marquis,^ ^ pp. 26-27. 

The major conclusion from this observation 
study was that the original hypotheses about the 
effects of such variables as information and 
attitudes on the quality of reporting were 
probably wrong. The major causes of good and 
bad reporting are probably to be found within 
the inteiview itself, particularly in the behavior 
of the participants. It was unclear which 
variables determined the behavior of partici- 
pants. The interviewer could be responsible for 
setting the behavior levels the respondent could 
have primary respohsibility, both could share 
equal responsibility, or the behavior level could 
be determined by some other variable or 
variables. The data led to speculation on a 
procedure referred to as a "cue search" model of 
interview interaction. It may be that the 



household interview is a rather unique experi- 
ence for the respondent and that he really does 
not have a ^t of predetermined behavior 
patterns for it. The newness of the interview 
situation might mdce it difficult to generalize his 
asiocialed feelings, attitudes, and expectations* 
The respondent must look to the interviewer or 
to some other source for cues about expected 
behavior. On the other hand, the interviewer 
may be in som what the same situation. She has 
learned from experience that respondents are 
different: some will enjoy the interview and 
others will be annoyed by it, some will have 
trouble with certain sections of the question- 
naire while others will not. Therefore, tht 
interviewer will be attentive to subtle cues from 
the respondent to help her arrive at a strate^ 
for dealing with each particular intemew. 

This hypothesis implies that both interviewer 
and respondent search for cues from each other 
about appropriate kinds of behavior. This 
cue-searching process could account for the 
strong tendency of interviewer and respondent 
to behave at the same level of activity in the 
interview* This heavy reliance on cues from the 
other person to set the behavior pattern may 
also account for the fact that cognitive 
orientations measured in this study were not 
predictive of behavior or reporting level. In 
addition, the reciprocal cue -searching process 
may explain why this research did not determine 
-whether-one-person-sets-the-behavior-activity™ 
level and the other follows. 

Subsequent research has shown that changing 
the characteristics of intenaewer behavior can 
have marked effects on both the amount and the 
accuracy of health data reported by respond- 
ents. These studies, while quite limited, support 
the general interpretation of the findings of the 
first observation study i namely, that changes in 
response accuracy are most likely to be achieved 
by changing the interaction process in the 
interview itself. These studies also show that 
changes in interviewer behavior will often be 
accompanied by changes in both respondent 
behavior and reporting accuracy* They do not 
rule out the possibility that data accuracy may 
be significantly affected by the respondent and : 
other sources of variation^ but they show that 
the interviewer can have at least some beneficial ^ 
effect independent of other possible influences. 
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URBAN EMPLOYMENT SURVEY 
BEHAVIOR INTERACTION STUDY 

With the cooperation of the U*S, Department 
of Labor, Urban Employmenl Survey, the 
Survey Research Center conducted another 
study^ of the behavior of the intervievver and 
respondent in the household interview. This 
study by Marquis and Cannell differed in a 
number of ways from the observation study 
described above* Data on the verbal behavior of 
the inter\iewer and respondent were obtained 
through a tape recording of the interview rather 
than through the recording of impressions by a 
third'person observer. The tape-recording proce- 
dure substantially reduces data cbllection costs 
and allows a much more detailed and refined 
coding of the verbal behavior that occurs during 
the interview. 

In this study^ a cross-sectional sample of 181 
employed male respondents residing within the 
city limits of Detroit were interviewed. There 
were four interviewers— all of whom were whiter 
female, middle-aged, and residents of the 
suburbs. 

New Coding Schema 

The study employed a revised coding, scheme 
for interviewer and respondent Verbal behavior* 
The coding scheme omitted ^11. nonverbal 
behavior/ It also included more code categories 
-for^task-related— behavior:~several---categories== 
renected the way in which the question was 
asked, and more detailed codes recorded the 
way the respondent answered questions. Because 
of the research on the effects of interviewer 
reinforcement wliich had taken place between 
the first and second observation studies, a code 
for interviewer feedback and respondent feed- 
back was included in the new scheme. Several 
codes dealing with irrelevant conversation were 
deleted since they had not proved useful in the 
previous study, A summ^y of the new coding 
scheme is presented in table 28, Additional 
items derived from more recent studies end the 
tabic. 

Main Findings . 

Behavior balance,— OnQ of the main findings 
from the original observation study was that the 



behavior of he interviewer and the respondent 
was best described in terms of a "balance" 
modeL That is, if one person was engaging in a 
great deal of behavior, so was the other. This is 
in contrast to another pattern which might be 
expected: namely, that a low level of respondent 
behavior would be compensated for by a high 
level of interviewer behavior, and conversely, a 
high level of respondent behavior would be 
accompanied by a low level of interviewer 
behavior. In this second observation study it was 
possible to control, both in the questionnaire 
design and in the statistical treatment of the 
data, the number of questions asked over the 
entire interview. Thus it was possible to 
compute an index of interviewer behavior level 
and respondent behavior level per question. The 
ability to control the number of questions made 
it possible to remove one source of variation 
which might have accounted for the behavior 
balance phenomenon observed in the first 
observation study* The correlation coefficient 
between the amount of interview behavior per 
question and the amount of respondent behavior 
per question was .77. This demonstrates again 
the strong interdependence of interviewer and 
respondent behavior during the interview. It also 
indicates that the variables or parameters which 
have a causal effect on reporting quality are 
probably to be found in the behavior interaction 
within the interview rather than in the personal 
cha racteri stics ( e, g, , attitudes) of either of the 
participants. 

Question asking and pro 6 m^.— Because of the 
expanded coding scheme and because the 
interactions were tape recorded rather than 
coded during the interview^ the second study 
provided a much more detailed description of 
the kinds of verbal behavior that occurred 
during the interview. The descriptive data 
confirmed that those interviewing procedures 
for which the interviewers were trained, such as 
question asking and probing, were carried out 
effectively in accordance with accepted proce» 
dures. Interviewers asked the question in the 
correct manner more than 90 percent of the 
time* Respondents gave many answers which did 
not meet the objectives of the questionj and 
interviewers used many probes in attempts to 
get adequate information. The probes used were 
generally nondireciive; that is, they did not 
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Table 28. Sumfmry of nvliid eodlnf sghemf fer Intirvltwif and rfs^n^nt wrbal bihiviar used in the urbin tmployrmnt iurvty 

bthavlgr intirBctlon itudy 



Oodi itefn 



CofriGt qutstldn .... ^ ... . 

incomplete queitlon 

Inappropriste qufftion^ . « , , . 

Ineorreet quistlsn 

Rtp^t quistion . . . , ^ . . . ^ . 
OmitM qu^lon . . ^ ^ ^ ^ , . . . 

information prevlsuily obnlned , 

Skip pattern . . . . ^ ^ ^ « . < ^ , , 
Nondireative probe^ ........ 

Dirgetive probe^ .......... 



Oivei cfsrlff cation. 



Votuntgeri Inforffiatlan , 



Adequate sn^ir .. ^ ..... . 

Inadequati an^er # . ^ ^ . . . . 

Don't know aniwir ^ ^ . ^ . ^ ^ ^ 

Refumensvvtr ..... . »^ 

Other answer ....... ^ ^ 

flaboration « « « , , , « . , . « . 

Aiks clarifleation . . . . p . . . , 
See footnotes at end of table* 



IntiMe^^r jjahavlor 
Qutitlon eskid e^ntlally as writtin on the quHtionnaire 
Fart of a^uettfon corregt as far at It goti 

Question ^at siktd but should have been skipped due to a sk^ pattern 
DuifteOn in whi^ meaningful word(s) have been ahered or mnltttd 
Question that h^ elraady b^n asked and Is asked eorrtody again 

Quaition QfTiitted by mistake^ contrary to the questionnaire Instmstioni, and for vvhich the 
filev^t Information hat not iMn ob^lned by means of a pr^^ing question 

Question omitted beeau^ sufficient Information to c^e an ^aquatt i^^nse has pftvlously 
been volunteer^ by the rt^md^t in ansv^r to i prior question 

Quettlon omitted bacau^ of skip pattern pmwll^ by the questionnaire initryctlons 

A probe that neither suggas^ g ^mif te answer or cla^ of ansvvfri nor rastrleti the frame of 
referent of the original question " 

A probe that suggest po^tble re^onses or Implies that gomt answi!^ are more ae^pteble 
tfian othfi^ it rastrlcti the frame of raferan^ of the original question. 

Oh^es clarif lotion upon request of the reipondint regardlen: of wfiether the Information sup- 
plied li correct or incorrtct. Irtclud^ also rephr^tng or explanations of queitions. 

Voluntten Information rtlevant to the topte of the queitlon or Intervlevv. Includes transition 
statements. 



Re^ondefit behevjor 

An adiquiti rasponie to a oorrectly asked question that meets the objectives of the qyei tlon = 
as stated In the Intervlawtr's Marniel. Incorrect clarification doei not rule out the ocGur^ 
rence of an adequate answer* May also omir as the result of a probe, provided the response 
meets the qyestlon objeetive. 

An inadequate rt^on^ to a properly asked question that does not meet the question ob|ec^ 
tlves as stated in the Intarvliwtr'i Manual 

Respond to a correct question that Indicates thit the re^ondent does not knovit only If nor 
followed by an attempt to answer the qutstlori 

Verbal refuisl to answer question 

Reipsnse Cto an Ineomplgte or incorrect question or a re^nse to a probel that does not 
nmt the qu^tlon oblecilve 

Gives reason for a rtspon^ or pj^llet more information that requfrid for an adequate aniwir ^. 
and is falevani to the quistlon topic j 

Request clarification of a question or quiitlon objeetlve 
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Tablft^, iummary of revlitd 



coding sehemt for Intervif wer and rispondent virbai behavior used in tht urban t mpioYmtnt suwy 
bfhavior Interaction itudy-^CQn. 



Code ittfn 



Feedback* 

Ongoing fftdbaek, , . , 

Rfpf^ts answer . . . , , 

Irrelevant convar^tion . 

Oives suggtitlon , . « . « 
Polite behavior . , . ^ , 

Intirruptlon ^ 

Laugh. 

Other. , . . . . ^ ^ , . , 
Eictraneous mttregtlon . 



ilmultsneous answer . 



Description 



Bihavlor of both Intervjewer and respondent 

Sehsvior that Indioatfi attint!on« spprovil, unde ritindlng, or hew well the other pemn is 
doing; only if not i response to a question or a probe (excluding **Thanlc you") 

Ongoing feedback that Indicates attantion^ approvai, understanding, or a desire to Internpt 
while the other it talking Without suc^tssfuMy interrupting the speodi of the other person 

Repetition of re^on^ either Maotly or as a summary* or utlliiation of pnvioui re^nses for 
traniitlon to a new topic or for asking a question or a probe 

Sutaments unrelated to the question or ^neril field of the Inquiry. Generally rapport- 
building or personal rather thin taik^rlinted bthaylors. 

Suggests new kind of behavior that will enhanet, intenupt* or reiume tesk behavior 

Pellti tohavior or socially eK^ettd coyrtiilfs not ^^Iflcally relate to task and not in^ 
eluded in tht printed question on the qutstionnalfe (e.g.* "Plew*" *Thank you**) 

Swc^siful interruption, Tht other perasn must stop talking. B looks can't occur at tht end of 
a ^ntenM or at a piuse which might be consider^ the end of a qyestion. 



^Modified qyast}dn,-«-<-ii-i-^-F^ 
Alttrnatives Incomplete ^ . . . 
infers answer # # , . ^ . . . . . 

"Anything el^" probe . . . . . 
Invents task^riented qyestion. 

Additional response. 
. Closure. < ^ ^ . . . . 



Audibit laughi chucNIej or snicker that may Indicate humor, tension, or ridicule 

Any signif leant behavior not el^where codtd or unintelilgiblt verbal bthavior 

Interaction of either interviewer or rt^ondent with a third person during the Intarvlew 

ADDITIONAL CATfGORiiS^ . 
Lntejyiewer bthavlgr 

-Question worded •m#7r/8//|«' as written but vvith unlmp^ — — — 



Falis to read all or some of rt^onM alternativeidfcae/f^ interrupted by re^ondent 

Omits question because Inttn/ltwer can infer the ansvver even ttjough it hai nmr been sated 
ejcpllcltly by respondent 

ipeclal case of nondirtctive probe 

Invents new question to gain or confirm Information ne^^ry to follow iklp initructfoni 

Respondent beheylor 
Given for e^h adequate answer beyond the first one for open^nded questions 
Respondent Indlutes no more Information to give on open^nded qyettlon 
Reiponddnt answers vvhile Interviewer is talking 



^ A probe is a qu^lon or statement used by the interviewer to elicit further information, ft Is a creation of the Interviev^r and Is 
not on the questionnaire. 

^Additional categories have been deri^^d from more recent studies^ 
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sug^st to the respondent any particular 
answers. This kind of probing, according to the 
theoryi helps to avoid the introduction of 
interviewef bias. Furthermore, the data suggest 
that the probing theory, with its emphasis on 
nondirective as opposed to directive probing, is 
correct. In tids study interviewers were much 
more successful in ^tting adequate answers 
after nondirective probes than after directive 
probes. These results are tentative however, 
since it is possible that mtervi ewers used 
nondirective probes when they expected that 
the respondent would have no trouble in giving 
an adequate answer and used directive probes 
only when they antidpated a great deal of 
difficulty in getting an adequate answer. Natural 
observation studies of this type are subject to 
this limitation on inference. Experimental 
studies are needed for a more refined andysis of 
the actual cause-and-effect relationships* 

Feedback and nonprogramed behavior.— 
Another major finding from these data is that a 
large proportion of interviewer and respondent 
behavior is nonprogramed behavior. That is, 
much that goes on during a household interview 
is not considered in typical interviewer training. 
Two sets of data illustrate this point. 

One discovery w^ that interviewer feedback, 
the interviewer's verbal reaction to the respond- 
ent's answer (such as "O.K.,'* *1 see^" *^Good,"), 
occurs very frequently. In fact, in this study 
interviewer feedback accounted for about 23 



Table Prdbibnfty of interviewer fetdbick foltDwIng reipend 
f nt bthiviofi, by dategory of r^^ndtnt behan/for 



percent of°aII^nt^iewer HeKa^ codeHTTTHF^ 
effect which interviewer feedback may have on 
the accuracy of these data is discussed in detail 
in the section "Use of Feedback To Increase 
Accuracy," of this report. These observational 
data indicate that feedback is very frequent and 
is probably used in a way that is nonproductive, 
or even counterproductive, of good data* 
Specifically, the data indicate that positive 
feedback statements occurred just as frequently 
after inadequate mswers as after adequate 
answers/ Most surprising was the finding that 
positive feedback statements were used over half 
the time when the respondent refused to answer 
a question. The probability of the interviewer 
using a feedback statement after nine different 
kinds of respondent behavior is shown in table 



tottgory of respondtnt behavior 



Adequate answif 
fnadiquite ans^p 
"Don't know" aniwir 
Rifuia! to mswir 

Other answer ^ 
Elaboration 

Riptati answer . . . . 

Glyes iugg^ion 

Dthir behavior (not eiassffled elsewhere) 



Probabll- 
fty that 
Intir' 
viewer . 

ftedback 
follov^ 



J4 
AB 

,34 
^0 
M 
,33 
.21 



The significance of the pattern of feedback 
use demonstrated in this study is not entirely 
clear, but reiults from this and other studiei lead 
to some hypotheses. In another section of this 
report several experimental studies are described 
in which different ways of using positive feed- 
back statements were tested (lee "Use of 
Feedback in a NoneKperimental Interview," 
this report). These studies show that if 
positive fi^edback statements were used by the 
interviewer only after the respondent has given 
an adequate answer, more accurate data were 
reported than when no feedback statements : 
were used. The data in table_2 9_indicat€_that theJ 
interviewers used feedback in a random fashion 
or when they felt some tension was developing 
or about to develop/ Neither the effects of 
random feedback contingencies nor of tension- 
reduction contingencies have been evaluated in 
the household interview setting* An educated 
guess at this point is that these strategies are less 
productive of accurate reporting than the usual 
laboratory strategy w Wch provides verbal 
reinforcement only after desired respondent 
behavior. Further research is planned in this 
. area, ^ 

Another technique for determining if there is 
more to the personal interview than asking 
questions md giving answers is to divide the = 
behavior data into two parts i (a) the average 
amount of behavior needed to get an adequate 
answer to a question, and (b) the average ^ 
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amount of behavior which occurs after an 
adequate aniwer and before the next question* 
Data in this study show that, on the average, 
one-third of all behavior occurred after an 
adequate answer and before the beginning of the 
next question. Computer programs are being 
modified to explore in greater detail the kind of 
behavior that takes place after an adequate 
answer. Although these results are not yet 
avalable, it is possible that this "extra" beha^dor 
may represent a large potential either for bias in 
the data or for cues which lead to even more 
accurate information* 

Effect of type of question on behavwr.-^Th^ 
behaviord data from this study make it possible 
to explore how different kinds of questions 
result in different kinds of behavior patterns. In 
his doctoral thesis, Thom^ dcKoning*^ 
dassifled questions on two independent dimen' 
sions: open-closed and fact-attitude. An open 
question was defined as any question to which 
the respondent must formulate his own answer, 
while a closed question was defined as one to 
which the respondent might either anwer "Yes" 
or "No j" or respond according to the alterna- 
tives supplied in the question. DeKoning*s 
closed question was similar to what others call a 
forced-choice question* The dependent variable 
was the average number of behavior codei 
assigned per question* Results which arc 
summOTEed in table 30 indicate that, as might 
be expected, there more behavior recorded 
nforT>penT[TOstioi^tffl 
When these data are split into specific behavior 
categories j it appears that intervieweri probe 
about three times as often and pro\dde about 
twice as much feedback for open as compared 
with closed questions. On the other hand, the 
respondent is about six times as likely to give an 
unacceptable answer to an open question as to a 
closed question* This pattern of results suggests 
that open jquestions present the respondent with 
more difficulty in meeting the objectives of the 
question. This conclusion is supported by data 
^ presented by "Question Length and Reporting 
Behavior/- in another section of this report, and 
by the results of an experimental study by 
Marshall, Marquis, and Oska , Open ques- 
tions require the respondent to retrieve informa- 
■ tion ; from memory with a minimum of 
stimulating cues/ On the other hand, closed 
h questions mvolva only recognition memory. 



TaWe 30. Rate Of resordtd bthtvi^r for open and clesed qus» 
tienSj by typt of behi^Hor 





Rate of be* 




hsvfor ptr 


Type of bihaviQr 


que 




O^n 








qUM- 




tloni 


tlor^ 


Intirvfewer bthavfor 






Total 


3.3/ 


1.13 


Corrtct qu^lon ^ ............. , 




0.91 


Probing , ,1 ............... , 


1.05 


0.32 


Feedtek . * . ^ . . . . . . ........ ..... 




0.46 


Othtr behavior . 


0.75 


0.44 


Reipondint b^gvlor 






Totil 


2.78 


1.65 


Adequate aniwer . . ................... 


.84 


M 


Unacseptabis answtr {Inadequatt, don't 






. knmig rffUial) , 


M 


.06 


Othtr answer ............... ^ ^ 


.63 


*35 


ilebOritlon 


M 


.24 


Other ^havror 


.B4 


.37 



That is, all the reipondent is required to do in 
response to a closed question is to decide 
whether the stated cham ctgjistic isjruj^^^ 
good or bad. While such questi^^^ have, 
advantages of clarity and ease of recaU^ it i^^^^ 
often necesswy to ask many of them in order to 
cover the same material as is covered by one 
open-ended question, 

DeKoning*^ also showed ^ t^^ there is a 
higher behavioral level in getting m answer to an 
attitude question than , to a question of ftct. 
However, the differences w not quite so l^a 
as for open and closed questionSi For attitu^^ 
questions inteiviewers are more likely to probe^ 
to give feedback, and to enga^ in inrelevant 
behavior and laughter than for ftct questions* 
Respondents are more likely to give unaccepta' 
ble answers, to ask for clarification, and to 
elaborate upon their answers when responding 
to attitude queitions, Theic data suggest that 
attitude questions are somewhat mom difficult ' 
and cumberiom e to handle than are fact 
questions, but this difference is small rad may 
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be due to other variables confounded with the 
attitude-fact distinction. 

Diapiosing specific question probtems.^One 
of the intriguing motives for using the behavior 
coding technique is to arrive at a lystematic 
evaluation of the adequacy of specific questions 
as they appear on the interview schedule. This 
kind of evaluation procedure may be very useful 
in the pretest phase of questionnaire construc- 
tion. Social science strives to be a scientific 
discipline, but the procedures used by social 
scientists to develop and validate questions and 
questionnaires are generally crude. One usually 
sends a group ot interviewers into the field with 
the questionnaire developed in the office. There 
is then a meeting (or series of meetings) during 
which interviewers and the researcher discuss the 
questionnaire. One hears familiar statements 
such as "This question seems to work well/" or 
"This question seems to do what we want it to 
because we have the distribution of responses."' 
The interviewer might say, "I don't think the 
respondents really understood this question,'" or 
"This question irritates people." It is on the 
basis of such highly subjective evaluations that 
questionnaires are df vrloped. 

In preliminary work done tV*r the U.S. 
Department of Labor, the fa^hav^or-codiii^^ 
method has shown considci.\!?ic promise for 
evaluating some aspct;t.^ y ^wiest • iterviewing. 

There seem to be ^hi ;c JJ?*Js of problems 
with questions: (a) fhoie atirfbutable to the 
"int e rvi e we rV ( B )^t Kosc i iiar Te f i< ct^Te¥pd1iy eiit ^ 
difficulty, such as failure to understand the 
questin.'i or trouble in recalling information, and 
(c) problems caused by the questions them- 
selveSj such as poor syntaXj **skip" ins^fMCtions 
that are difficult to follow^ or ubs^.ure 
placement on the interview scheduiv A 
small-scale attempt was made to trace quiif^liijn 
problems to one or more of these sources by 
using a small number of the codes obtained in 
the behavior observation study. 

For example, those questions that were asked 
incorrectly by the interviewers at least 15 
percent of the time were identifiedp A question 
was coded as "incorrectly asked" if important 
words or phrases were changed or omitted.- The 



list of incorrectly asked questions revealed 
several items which contained parenthetical 
phrases, others which contained difficult syntax, 
and still others which were extremely cumber- 
some to handle in verbal form. The first set of 
questions pointed to the fact that during 
training, intervie%vers had been given incon- 
sistent rules about handling parenthetical 
phrases. The second group of questions, 
containing awkward syntax, pointed to a 
problem that has been overlooked by many 
qu^jstionnaire designers: When questions are 
extremely long and complex, respondents often 
interrupt at the end of a clause to answer 
without allowing the interviewer to finish the 
question. 

Another set of 39 questions was identified as 
having been answered inadequately more than 
14 percent of the time even though they were 
asked correctly. From this list there appeared to 
be two reasons why a question would receive an 
inadequate answer code a high percentage of the 
time^ (a) the respondent %vas Unable to answer 
because the required information was not easily 
accessible from memory* and (b) the interviewer 
could not discriminate between an adequate and 
inadequate answer, and therefore mistakenly 
accepted the inadequate answer as meeting the 
objective of the question. 

Another analysis of question problems was 
tried in which two kinds of interviewer omission 
codes were combined to show different kinds of 
quest iOT^pfoblel^^ c u f t hli t^ialy Siil"is^ 

as follows:. 



Nature sf probltm 


Code (nP 


Code r}2 


Interviewer error 


High 


High 


QuestionnalrD redundancy , . , , . , , 


High 


Low 


Skip pattern or forrnat problern . . . 


Low 


High 


No probjem 


Low 


Low 



-Code N indicates a queitlon was omitted because the mswer 
was already given. 

-Code * indlcatei that the question was omitted by mistaki, 

NOTiS: "High'* Indicates the question wis omitted 10 per- 
cent of the time or more. - 



"Low" Indicates the queitlon was omitted lesi than^lO per' 
cent of the time. 



^MoFC stringent criteria wcfc Use*/' iht last section of the 
qucitiomiaira which contminsd mostly air-' ide questions. 



There were 27 questions identified as being 
oniitted many times, either because of error or 



because the interviewer thought the answer had 
already been given. The data suggest that 
omission problems like these might be overcome 
if interviewers received better instruction as to 
what constitutes an adequate answer* Thirteen 
questions were identified as belonging to group 
threcj questions which were omitted often by 
mistake but were not skipped because an answer 
had already been obtained. This finding also 
suggests the need for better imcrviewer trdning 
since these questions are often skipped because 
the interviewers assume they have been an- 
swered adequately through previous questions— 
when indeed they have not* 

On the other hand* better interviewer training 
concerning the objectives of each question may 
not entirely solve the omission problem* Other 
data indicate that omission rates are above 10 
percent only when a question is to be asked of a 
subsamplc of respondents* Questions which 
must be asked of all respondents are almost 
never subject to high omission rates* Of the 71 
questions in this intemew which were to be 
asked of all- respondents^ only 1 was omitted 
more than 10 percent of the time, while of the 
102 questions to be asked of subsamples of 
respondents, 55 (54 percent) were skipped more 
than 10 percent of the time, either by mistake 
or because an adequate answer had already been 
obtained. Thus, omission problems may be 
traced to skip instructions and other subsamf 
plin g techniques * While these procedures are 



often necessary^ the questionnaire designer 
should be aware of the potential for interviewer 
omission error whenever subsampling techniques 
are used. The subsampling omission bias may be 
especially acute in an interview such as the one 
tested, in which skip patterns occur frequently* 
Other procedures to identify question prob- 
lems have been or will be tried* The possibility 
of obtaining systematic data on question 
problems in the pretest phase of a survey study 
remdns intriguing* Much work is still to be done 
in devising procedures relating to question 
design. The Survey Research Center methodol- 
ogy program is working to develop additional 
kinds, of logical analysis of question problems, as 
well as to reduce the cost and time involved in 
obtaining such data/ When some of these latter 
problems are solved, other survey research 



agencies should be encouraged to eKperiment 
with these appropriate procedures* There are 
enough eKperimental and observation studies 
now in the literature to indicate that question 
variables such as structure and content are 
important determinants of data accuracy in 
survey intemews* While many factors may 
potentially affect data accuracy, the question 
variables probably have a much greater potentied 
effect on overall accuracy and completeness 
than does any other single class of variables, A 
thorough understmding of how question con- 
struction and question content affect data 
accuracy should do a great deal to advance the 
usefulness of the survey interview for reseaLrch 
purposes* 

Effect of respondent age and race on 
fi^Aayiofp— Social scientists and those responsible 
for the conduct of cross^sectional sample surveys 
often hypothesize that the demographic charac- 
teristics of the intemewer and the respondent, 
such as age, race, education, and income, will 
have some effect on survey data accuracy* For 
example, earlier research has shown that white 
respondents are reluctant to admit prejudice 
toward blacks when the interviewer is black* 
Such results are interpreted as being reasonable 
in terms of cultural norms concerning prejudice. 

The question remmns, however, whether the 
results of these studies are applicable to the 
reporting of all information in all survey 
intemews. For examp_le8_the^re__is eviden ce to 
suggest that the accuracy of data obtained about 
such things as physician visits or financial facts 
does not differ by race of respondent. This 
second SRC observation study included an 
experimental design which provided for the 
investigation of interaction patterns by respond- 
ent age and race. White middle-class female 
interviewers interviewed - employed male re- 
spondents. There were four experimental groups 
of respondents^ (a) white, 18-34 years (N^47); 
(b) black, 18-34 years (N=44)| (c) white, 35-64 
years (N=43); and (d) black, 35-64 years 
(N=47), 

With respect to the effect of age on behavior 
during the interview, the data confirm the 
results of the first obseivation study. Older 
respondents were much more likely to engage in 
a large amount and wide variety of behavior 
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during the interview* The proportion of their 
behavior devoted to good task performance was 
much lower than that of younger respondents* 
In addition, when interviewing older respond- 
ents, the intervieweri displayed high frequencies 
of a variety of behavior* Thus, an interview with 
a younger respondent was quite different from 
an interview with an older respondent* The 
former appeared to be task oriented while the 
latter was characterized by a great deal of extra 
behavior which may have resulted in keeping the 
interaction at a relatively tension-free leveL 

The effects of respondent race on the kind of 
behavior shown in the interview were not clear* 
Two attempts have been made to analyze and 
explain these data,^ ^ and both produced a 
somewhat similar set of inferences based for the 
most part on nonsignificant statistical trends. 
The overriding conclusion is that when age was 
controlled, the effect of respondent race on the 
kind of behavior in an interview was not marked 
for the kind of information contained in the 
interview* However, the data do suggest that if 
there was a race effect, it was in the areas of 
interviewer probing behavior and respondent 
inadequate answering behavior* Although the 
differences were not always statistically signifi- 
cant, it appeared that the proportion of 
inadequate answers (answers which normally 
require probing) was higher among black than 
among white respondents, and that interviewers 
^prob.ed_jnore„wlth_bJack_than=^J^U^ 
respondents* Also, black respondents tended to 
give more **don" t know" answers, repeat more of 
their answers, and ask for clarification more 
frequently than did their white counterparts* 
These racial differences are very sm^l but may 
indicate slight differences in difficulty vsdth the 
questions^ The pattern viewed as a whole does 
not indicate any active resistance or lack of 
motivatiori to cooperate* 

There was a slight tendency for white 
respondents to exhibit more abiUty to give 
adequate answers than did their black counter- 
parts, and interactions with the female inter^ 
viewers seemed to be less task oriented among 
white respondents. For examplej white male 
respondents engaged in slightly more polite 
behavior, feedback, and elaborations than did 
black males* White males also seemed to show 



more resistance or more dominance while 
performing their task, as indicated by a slightly 
higher percentage of refusals to answer, 
suggestions to the interviewer, and unsuccessful 
attempts to interrupt the interviewer. 

In summary, the age effect was found to be 
fairly reliable. It wm quantitative rather than 
quditative* Older respondents in comparison to 
younger respondents engaged in a higher 
percentage of almost every kind of behavior 
except providing adequate answers and making 
requests for clarification* The race effect was 
much smaller than was the age effect. Black 
respondents showed a pattern of behavior 
characteristic of well-motivated performmice on 
a difficult task. White respondents seemed to 
have an easier time at the t^k, interacted more 
smoothly with the interviewer, and showed a 
slightly greater tendency toward dominance or 
resistance. It seems likely that whatever 
differences exist may reflect variables such as 
educational backgrour^d of respondents rather 
than race as such. 

Interpretation problems—The cHuii:-;:*^^' in 
trying to interpret the nonsignificanv ^l^%ift^c^$ 
between the two racial groups points to an 
apparent problem or handicap in th^ citrt^nt 
behavior obi :.rvation scheme.^ The problem 
seems to be that the readily coded behavior 
categories such as "asks question correctly^" 
"refuses to answer," and "laughs" are difficult 
to define in an abstract sense^ So^ial scientists 
are accustomed to dealing with abstract 
concepts about human interaction, such asr 
"shows hostility," "is annoyed," "interacts 
smoothly," or "is having conceptual difficulty/' 
At an even higher level of abstraction these 
concepts might be* "is ingratiating," "shows 
lack of rapport," "enjoys the interview," or "is 
motivated." A point to be made in defense of 
the observation technique and its scheme of 
categorizing behavior into sm^l units is that 
extrapolating behavior codes to a higher level of 
abstraction does not really provide much more 
meaning to the data* The theories of human 
interaction to which some have attempted to fit 
the existing data have not themselves been V 
validated to any great extent. Interpreting the - 
present data in these frames of reference will 



■Jprobably not y ; Thus, the next research steps 

1?^^^?^^^ P^^^^* ^ several kinds of studies which record and code 

iS- ?!^^ the problem of the behavior that bkes place in an interview and 

f?tt?}^uting meaning to the obseryationa] data which, in addition, obtain independent verifica-" 

^ rnay He carried out in a different way. If it is tion of the accuracy and cbmpleteness of , the, 

^recognized diat the problem of the survey data the respondent has reported in his aniwers. 

5 researcher is to obtain complete and valid data. Correlations between the accuracy meaiures arid 

> the strategy for assigning meaning to the various the behavior measures would then yield 

^behavior observation codes becomes ftirly clear, significant insights into tlie meaning of the 

j Empiricd research is needed to establish the behavior codes or combinations of behavior 

■:cause-and-effcct relationship between the occur- codes. 

rence of different kinds of behavior or patterns 

:of behavior and the validity of data reported. 

INTERVIEWER PERFORM ANCE DIFFERENCEr 
SOME IMPLICATIONS 



1^ Most survey researchers believe the adage that 
practice ma^es perfect, or at least makes for 
improvement. Thus, they expect seasoned inter- 
^viewers when compared with less experienced 
ones to be more skilled at adapting to various 
^interviewing situations, more at ease in interact- 
j^irig with respondents from various social classes, 
and more proficient in using nonbiasing proce- 
dures. In ten^iews conducted by an experienced 
^staff ; usually present fewer: problems in coding; 
the ^^responscs are clearer and more adequate to 
^he objectiveSj contingencies are followed more 
strictly, iandj with a minimum of omitted^ues- 
tions, there are fewer noncodable replies. 

Data from methodological studies which were 
recently examined raise questions about the 
poritiye effects of experience in interviewing, 
Since j the analysis of interviewer behavior was ; 
not planned as part of these studies, the research 
designs are inadequate to produce conclusive 
flndings^ but are sufficieritly intriguing to en- 
courage further study; As an iricidental analysis 
in one study, the data on failure to report 
known hospitalizations were tabulated for each 
item separately. These data showed a surprising 
trend: The larger the number of interviews taken 
by a single interviewer, the fewer the hospitaliza- 
tions reported by the respondents. A^^ 
random assigrimcnt of interviewers was not 
rhade, and the results therefore might reflect a 
iifference in types of respohdents intemewed 



by each intemewer, the finding was sufficiently 
interesting to raise questions about the positive 
effects of experience in interviewing and to lead 
to an analysis of interviewer performirice-^ia^^^ 
other studies. 

The first SRC studyV^ w^ designed to 
investigate the underreporting: of-Kbipitaliza- 
tions* A sample of approximately 2,000 hospital 
records was selected from patients who were 
discharged from a hospital within the 12 months 
preceding the interview. The sample was taken 
from hospitals located in counties that were k 
part of the r Health Interview Survey (HIS)/ 
regular national sample. The family name and- 
addr^^^ of the person discharged were given to 
the ^.areau of the Census interviewers who 
regularly conducted the interviews for thejnter- 
view survey in that county. Twenty-seven ex- 
perienced Bureau of the Census interviewers 
were included in this study. All interviewers 
working in the areas in which sample hospitals 
were located did some interviewing* Because the ; 
number of sampled discharges varied consid- 
erably by county/ interviewers received varying 
numbers of sample addresses each week| i 
The questionnaire and interviewing proce- ' 
dures used were identical to those of the regular f 



■^F Of detaili of the sample, the procedures used/ and the 
flndinp ef tin % HcaJth Interview Survey, see reference 17* . : 
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Health Interview Sun^eyr lntervieweri were in- 
formed by the censui field office that they were 
to undertake a special study and were given 
ipedar sampUng instmctions. They were 
that the regular questionnaire mid interviewing 
procedures were to be followed. Interviews were 
conducted with each adult member of the 
family found at home at the tirhe of the visit* 
For those not at home and for all children, a 
proxy "responsible adult" was interviewed. 

While the interviewers were aware that this 
was a special study, the purpose was not 
divulged. In order to attenuate the number of 
hospitalizations reported, a sample of names and 
addresses drawn from a telephone directory was 
added. However, interviews at these addresses 
were not used in the analysis. 

Following the usual procedure, each inter- 
viewer was given a weekly assignment which she 
was to complete during that week. The inter- 
viewing extended over 3 months, and the inter- 
^dew reports were matched with the hospital 
disch^ge records. Table 31 shows the rate at 
which hospitalizatiQns were underreport^ by 
the number , of interviews taken by groups of 
interviewers. The data show a tendency for the 
rate ' of underreporting of hospitalization to 
increase as the number of interviews increases. 



/^^hf reader Is cauUoncd not to inurpret the Ogures in this 
pa^r as a measure of net reporting bias, since only underreport^ 
ing is included. An esdfnate of the rate of overreporting U not 
possible with this sample^ as it requires a different research 
deilgn* 



The rank order correlation of the number /of;:: 
interviews taken and the failure to report the ^, 
hospitalization is very high, ■ ' 

Attempts to understand these results- by ^j^^ 
looking for differences in characteristics J pf ?^ 
hospitalizations were not fruitful. The overall - 
response rate for this study w^ 95 percent; thus, ^ 
differences could not be attributed ; to low ^ 
response. Interviewers were given a weekly -r 
assignment depending on the number of sample 
disclmrges in their county or that part of the^:;:: 
county in which they worked; thus they had no^^^^~ 
choice in the number of inteTOcws ^ be ^ 
conducted. ., ■.■ 

The most tenable hypothesis to explain these ; 
findings is that interviewers lost interest and , 
enthusiasm for the work. It may also be that - 
interviewcri had some feeling that an aim of the ^ 
study was to check on their : performance ^ ; 
aithoujh much reassurance was given that this - 
was not the case and they gave no indication of 
such concern in intendews conducted with them. .; 
after the completion of their interviewing. The' 
data^ indicate_ that, _ whatever ^ 
interviewers behaved differently in the earlier 
interviews than in later ones. 

Data from other vaUdity studies of hospitali-:^ 
zations and physician visits were then analyzed . 
to see whether the pattern was. replicated and to 
gain greater understanding of the finding. Data 5: 
were available from another study on under- 
reporting of visits to physicians during the 2 
weeks preceding the interview* A sample wai^^^ 



Tabli 31. Numbar of ricorded hoipitsrdl^hirgei, mt dian numter ©f rtperted dlichargti par interviewer, and rata of underreporting, 

by nurnbef of Inteiviivvs per intefviiwirr iun^^ : 



Rank order of Interviawiri by numbtrs of Intervltv^ taken 
(lowest to higheit) 



1-5 . . 

6-10 . 

IMS . 

21 '25 , 



Recorded 
diicharges 



54 
237 
330 
506 
897 



Median number 

efdlsehargei 
reported par 
mterviawtr 



4 
4? 
iS 
100 
137 



Rata of 
undirripoftlng 
discharges per . 
mtervitwef ^ 



Medten Mean 



0 
10 

12 
IB 
12 



io.b: 

12.4 
-15.6 
^12.5 



Difference between f ii^t five and laft five interviewers significant at p ^ ,05. 
Sou rser reference 1 7. 



^^^^^'^^^"^ records of a large iubscriptidn 
• medical: care plan in t^^ Detroit area. A sys- 
®*pipJ^^^ clinic phYsicians was 

drawn weekly for 8 weeks. With each week 
: compriiing a random sample of those visiting the 
cUnic'du][^ that particuto^^ 
;*^*^^^^^^^^^r^^onductedi Since mmy respond- 
v^t^ ^^d multiple y interviews ac» 

counted for a total of 403 visits for the 2 weeks 
■ preceding the week of interview. 
; : Ten interviewers were hired by the Survey 
Research Center for this study. All were com- 
paratiyelyr inexperienced. worked 
briefly on a methodological study; others had 
workedr for a short time on the U J, decennial 
:^^nsus, but most had no previous interviewing 
ekjperience. 

r Special training s manuals and material were 
prepared and a supervisor with several years of 
interviewer training experience conducted the 
^^^Sf^^^^^d by two other experienced field 
supervisprs. Three weeks of training were com- 
pleted before the actual interviewing started. 
Classroom training and field assi^ments were 
conducted during the first weekv and during the 
second week each interviewer was observed as 
she ^4, conducted interviews at nonsample 
addresses. The third week consisted of inter- 
viewing assignments which interviewers thought 
were part of the regular study, but which were 
actually addresses from the telephone directory. 
During the field work, questionnaires were re- 
viewed in the office and errors were discussed 
vVith interviewers, 

: The:que^^ to that 

used in the Health In tervie w Survey. The ques- 
tio>is about visits to physicians were as follows: 
'Last week or the week before, did anyone in 
the family talk to a doctor or go to a doctor's 
iffice or clinic?" Three probe questions were 
idded for specific types of visits which respond- 
mts might consider to be outside the scope of 
he: question: (a) "At the time of this visit was 
he doctor asked for any medical adyice for any 
)ther member of the family?- * (b) "Did anyone 
the .femily get any medical advice from a 
ioctor oyer the telephone lajt week or t^^ week 
»efore?'V^c) -bid anyone in the fa^ a 
lursc or technician for shots. X-rays, or other 
reatment last week or the week before?*' 
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In this study, although interviewers were not ' v 
given random assignments, the total sample for 
each week was an independent random sample; - - - 
of visits. It is, therefore, possible to m^e ; 
comparisons of undeireporting rates for all ~ 
interviewers for each of the 5 weeks in which 
intervievying took place (see table 32)* : vr 

Table 32 shows a significant increase in :A 
underreporting over the B-week period of the^' r v^ 
study. The difference between week one and • ^ 
week five is significant at the 5-percent HeveL ' 
There is a decrease in validity of reporting over • ~ 
time. This finding tends to confirm the results 
shown in table 3 L In the first study reporting : 
got progressively worse as the number of inter-_ r 
views taken by an interviewer increased, and in 
this study underreporting increased as time 
progressed. - 



Table 32, Number of recorded physician visits and percpM ni>£ > k 
reported in interviews, by week of interview: S urvsy Re--V -tl- .'A? 
search Center ~ " ■ " ' . ' 





Interview week 




Ricorded 
' visits:^ ;^ 


Percent*' 
poriitt'i *^ * ' 




1 . 






■ ' 78 


18 




2 , 






102 


21 ' 




3 . 






95 


■ ' 23 " 




4 . 






73 






5 , 






55 


29 





iourcer reference 17. 



However, even though underreporting in- ■ .. 
creased as time progressed, the rate of under- ^ 
reporting was actually lower in the second study ^ 1; 
when a comparison was made of the rates of 
underreporting by the number of interviews 
taken by each intemewer (see table 33)^ This ^ ^ S 
finding contradicts the conclusions drawn Jfom"^^^^^^^^ 
the data of tables 31 and 32^ An explanation 
may be found in the reasons vvhy inteiviewers 
conducted more or fewer interviews in the two 
studies and in the way the assignments were ; ^ 
carried out. In the hospitalization study, inter- , 
viewers were given weekly assipiments by. the ^S? 
office and had little to say about the number -^^S 
given because each interviewer worked only in 
one geographic area and had to take all inter- 
views in that aa'ea. Therefore^ some had a heavier 



- Table 33* Number of fteordtd physician visits and peraent not 
reported fn interviews, by number of'' weekly Intirviews 
' ; conducted per intervievver? Survey Researsh Center ; 





- 

Individual number of 


Recorded 


Ferc^nt 
not re- 
ported 




Inten/iews per week 


visits ' 






26 


31 


2 . . 




2i 


38 






28 


29 


4 . . 




3§ 


20 


i . . 




39 


28 


6 . . 




43 


28 


7 . . 




44 


2i 


S . . 




49 


^- 20 


© 




4i 


20 


10 . / 




64 


14 



Sourcer referent 17. 



wacldy load than others, regardless of their 
wishes. In the physician visits study^ all inter- 
views were taken in a single area, and each 
interviewer could take interview! in any part of 
the area with little increase in cost* Within 
UihitSjintemewerswel^ permitted chbdse^he 
number of interviews they wished to take each 
week. Thus, the choice rested wth the inter- 
wwers. Their choices may reflect n a greater 
involvement or interest in participating in the 
study s or, since interviewers were paid on an 
hourly basis, it may reflect a desire to earn more 
money* Thus, the difference in underreporting 
rates between interviewers in the two studies 
may reflect a difference in their motivation* : 
Since interviewers were generally consistent in 
their choice of a large or smdl number of 
interviews each week> the original finding of m 
incre^e in underreporting is understandable 
again in - terms of interviewer interest and en- 
thusiasm* Those with low motivation for the job 
lost interest early, and even the enthusiasm md 
reporting accuracy of more highly motivated 
interviewers waned over time in the course of 
the fieldwork. 

Another validity study of hospit^zation 
reports by Marquis and CanneU^ ^ utilized three 
different field procedures and thus permits 
comparison of interviewer performance* For this 
study, which differs in several respects from the. 
hospitalization study reported earlier, a sample 



of discharges was ^elected from 18 hospitals in 
the Detroit metropolitan area. The design con- 
sisted of four orthogonal randomized latin 
squ^es. The mayor iources of variance were five 
interviewing weeks; five regions of the city; and 
three field procedures, one control and twOj 
experimental. This design provides a base for 
comparison of intemewer performmce that is 
less confounded with other variables than that 
of previous studies* 

The three fleld procedures were as follows: 

Procedure -4* --This was applied to the control 
poup^ ThiB procedure involved ^ essentially the 
same standard Health Inter^dew Survey question- 
n^re that was used in the other two studiesr 

^froceduf^ 5,— The questions and procedures 
were the same as in procedure A, except for 
hospitalizations* The reference period for 
questions on hospitalizations was a year and a 
half instead of a year, * Probe questions were ' 
added and special introductory statements were 
included.^ , . - . 

JVoc^dwre C.--The interview questionnaire 
was identicd tojthat^uiedin procedure A except 
that no questions on hospitalizations were asked 
during the interview. A form to be filled out by ^ 
the family was left with the respondent* 
Nonresponses were followed up by mail and 
telephone* 

Each intemewer w^ assigned to : two 
procedures* One group used procedures A and C; 
the other group used procedures B and C*- 
Twenty interviewers were employed (most of ; 
whom had very limited inter\dewing, experience). 
Inexperienced : interviewers were used so that 
they would not l^ow that the- various 
procedures were different from those ^ cus- 
tomarily used in the Health Interview Survey.^ 
The training was conducted by experienced 
trainers using standard intemew methods* There 
was one full week of trdning and^^^ractice 
interviews followed by field observation of each 
intemewer. A comprehensive mmual was: 
prepared ^that ""specifi^'^' all ^ techniquesr^'^TheT 



procedure also utilized a mail foUowup qu^tionnairc 
desipied to pick up fiffth^ hospioliEatioa . reports* The data 
presented here do not include results of the follo\^p and conilst 
only of reports pven in the interim^ 
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training was conducted in two groups and by 
two trainers j one for those using procedures A 
and G and the other for those using procedures 
BandC- 

Table 34 shows a pattern percentage of 
. underreporting of hospitalizations for each of 
the 5 weeks for each of the three procedures. 
Procedure A (control group) shows a pattern of 
underreporting much Ukc . that found in the 
study of physician visits* Reporting was poorer 
as the fieldwork progressed. Procedure B 
{experimental interviewer) showed a similar 
pattern with small differences. It may be that 
the effect of the experimental procedure was to 
diminish or eliminate the effects of time on 
intemewer performmce* Two factors, the 
additional probes and the supplementary state- 
ments to respondents , about the study^ may 
account for this effect. Procedure C (the 
self-administered form) does not show this 
pattern and would not be expected to since the 
interviewer merely handed the hospitalization 
form to the respondent, asking that it be 
completed and mailed in. Because the design 
called for approximately equal numbers of 
interviews, per interviewer, it is not possible to 
comp^e reporting rate by number of inter\dews 
conducted. 



Table 34. Femnt ef recorded hospitalizations not reported In 
intimews when proctdurts A/ and G were used, by week 
of mteryiew; Survey Reseergh Centtr 



Week of the interview 


Hospitalization 
interviewing procedure 


A 


B 


C 




Percent of hoipitili^ 




f ations not reported 




13.7 


8.3 


14.4 




11.0 


8.6 


1i.O 




1§.8 


9.2 


21.2 




22.1 


S.7 


10.S 




23.7 


10.0 


16.1 



Source: reference 18. 



TTiere is another bit of e^ddence in this study 
rhich supports a motivation hypothesis. Each 
itcrviewer used two procedures r procedure C, 
Wch involved the self-administered report of 
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hospitalizations left with respondents to com- 
plete and mail, and either procedure A or B, 
both of which obtained a report of hospitaliza^ 
tions in the interview* The average underreport- 
ing rate obtained for each interview cmn be 
compared with the average underreporting rate 
based on the mdled return; The product 
moment correlation for the reporting rate by 
interviewer for the control procedure and mailed 
response procedure is *65; that between the 
experimental procedure and mailed procedure is 
-56, 

These relationships are surprising^ particularly 
because one reason for using a lelf-administered 
procedure was to avoid the interviewer's 
influence. The relationship isj of course, based 
on the performance of only 10 interviewers in 
each group. One interviewer who was singularly 
unsuccessful in obtdning reports of hospitaliza^ 
tions in either procedure contributed dispropor- 
tionately to one of the correlations^ However, 
these data indicates as do previous results^ that 
interviewer behavior varies md that this 
behavior affects responses. Inter^ewei3 who 
were more successful in stimuladrig respondenti 
to report hospitalizations during the interview 
were also more successful in stimulating 
respondents to better performance in filling out 
a self-administered form and mailing it to the 
office^.. . :. - . . . 

Another interesting behavdor pattern was 
discovered. The questionnaire for the study of 
physician \dsits contuned one main question and* 
three fgllowup probe questions desired to 
make .^ure that the , respondent's concept of 
physician visits was the same as the interidewer's 
and to stimulate recall of e^ily forgotten events; 
An overall 7-percentage-point improvement in 
reporting was achieved through use of these 
followup probe questions, but the data demon- 
strate that some interviewers used the probe 
questions quite differently thm did others: Not 
only Bic there large differences in the amount by 
which the probe questions improved reporting 
for different interviewers, but memingful ; 
patterns are present* 

The rates of miderreporting for the 10 
interviewers according to the total number of 
interdews conducted are shovm in table SBi TTie 
improvement in reporting through use of the 
probe questions is shown in the last column of 
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Table 35? Number of ricerded physician visits reported in intefvlevfl^, percfnt not reported when procedures A and B were used, and 

. percent of impfovament in feportlng rate^ by individual interview . 



Intin/iewer number 



1 . . . . . ; . . . . V , . . . V. . . . 

2 ... * .... . / . . . . . .... . 

3 . . . . 

4 . . , , . . . . . . . ......... 

B . , .............. . . . . 

©..,.... .... . . . . .... 

7 

8 ...... . . , . . . . . 

e . . .... .............. 

10 ......... - 

NOTE' Data previoysly unpublished. 



Number of 
recorded 
physician 
visits reported 



26 
28 
28 
3§ 
09 
43 
44 
49 
49 
64 



Percent not reported 



Main question 
only 
(A) 



69 
3S 
29 
20 
26 
28 
34 
33 
22 
20 



Main question 
plus probes 
(B) 



31 
3S 
29 
20 
26 
26 
23 
20 
20 
14 



Percent 
improvement In 
reporting - 
(A -8) 



39 
0 
0 
0 
0 
- 2 
11 
-13 
2 

a 



this table. When only the main question was 
used there was not a clear relationship between 
the number of interviews taken and the 
underreporting rate . (rank order ^ correlation- of 
^52). When the probe questions were used there 
was more of a tendency for interviewei^ taking 
ah inereased number of interviews to have lower 
reporting fates (rank order correlation of .8 3) * 
The most interesting finding was related to the 
effect of probe questions. Only one of the five 
interviewers recording' fewer than 40 sampled 
visits improved the reporting by using probe 
questions, but all of those who recorded over 40 
visits sHoweH^mprovement in reporting when 
the probes were used. The median rate of 
underreporting is within 1 -percentage point for 
the mmn question when the top five interviewers 
are compared with the bottom five interviewers. 
When ail the probe questions W£re used, there 
was no change in reporting fw- the first five 
interviewers, and a 6-percentage-point improve- 
ment in reporting for the five inteiviewers 
having the greatest number of reported visits. 

Two conclusions are suggested by these data. 
The first is that interviewers differ in the way 
they use questions and probes. Some apparently 
make little use of the probe questions, either 
failing to ^k them or asking them in an 
incidental manner. For four ■ interviewers the 
probes failed to elicit any additional informa- 
tion. In contrast, interviewer number 1, who 



seemed to have placed most of her reliance on 
probe questions, experienced a 39-perccnt 
improvement in reporting when she used the 
.probes. The training and supervision failed to^ 
obtain uniformity in the use of the probe 
questions. 

The second conclusion is that, except for the 
unusual behavior of in temewer number 1^: the ^ 
interviewers who had the largest number of 
sampled visits (who may. have been more 
interested and motivated) generally made better 
use of the probe questions. A major intemewer- 
difference in this study and the hospitalization 
study is the amount of experience in interview- 
ing and in using the Health Interview Survey 
questionnaire. For the experienced intemewers^ 
there was a drop over time in the reporting of 
known events, suggesting that additional experi-^v 
ence did not m^e for improvement: in: skill, 
since skill level was dready.high. The drop can 
be accounted for in motivational terms; the 
novelty of the studies wore off. In the physician 
visit study, the interviewers were inexperienced. 
Those who conducted a large number of inter- 
views improved, suggesting that they did profit 
from longer experience and did gain in skill. This 
finding plus the earlier motivation hypothesis 
suggest that interviewers who are motivated 
improve their performance with added experij 
ence. For the less highly motivated interviewers 
this relationship is not as strong. This interpre- 



; only on speculationp would help 

explain the conflicting results in the time 
■\ studies. ; ■ 

V One final study is relevant. In this study,** 
which did not have ralidity meaiures, the 
■ dependent variable was the number of health 
conditions and behaviors reported/ Since pre- 
vious studies suggested that the major problem 
r^pprting health events is underreporting 
; rather than overreportingi the working hypothe- 
: sis in this study was that high reports ict likely 
to be more accurate than low ones. The study 
had two experimental interviewing procedures 
and a control group. The control procedure wai 
fairly similar to the questionnaire used in the 
Health Interview Survey. Only the control group 
is used for this analysis. Because the experi- 
mental procedures required interviewers to fol- 
low rigid rules of interviewing techniquesp it was ' 
necesiary to institute special field supervisory 
procedures. Each interviewer w^ observed dur- 
ing her interviewing every week during most of 
the fieldwork. Attention w^ focused on inter- 
viewing techniques and each interview was dis- 
cussed with the interviewer. 

Average reports for the number of conditions, 
symptomSj and physician visits respondents re- 
ported for themselves and for other family 
members are shown in table 36. In contrast to 
data of other studiesj these data show higher 
reporting of events in the second half of the 
fieldworks for all but one item/ Self-reports for 
chronic and acute conditions and symptoms 
were significantly higher during the last 3 weeks 
of the study than during the first 3 weeks. 
: "Again a motivational hypothesis explains 
these findings* In this study, in contrast to the 
preceding ones^ attention was paid to the inter- 
viewer's performtace in interviewing* In pre- 
vious studies interviewers were rated on the 
quality of the completed questionnaire, while 
for this study the reward was for good interview- 
ing performance. It seems likely that both 
different criteria and greater personal attention 
led to greater skill and increased motivation in 
performing the interviewing task/ 

CONCLUSIONS AND DISCUSSION 

I - . .. _■ • ■ - . ... ■ 

: .^-.1 ' ■■ ■- ■ - i ■ - . .. ■-■ ■ 

The evidence from these data indicates that 
:he interviewers generally did not improve their 



Tibli 3fi. Average ffiqutncy of health feports during flFit 3 
w^ks end socend 3 wseki of the interviewing period uiing. 
pro^lures A and i, and Improviment in rt^rting during 
steond B^wmk period, by respondent ftatus end health event 



Average events reported 



Respondent status 
end health tvent 


First 
3 weeks 
(A) 


Second 
3 weelcs 
(i) 


Improvement 
during second 

3 weeks 

(S-»A) 


Reipondint reporting 
foriilf 








Chronic and scute 

cendltloni . , , . , 
Symptpn^ , , , , , , 
Phyildan viiiti . . / . 


1.S7 
4.88 
0.34 


2J3 
e.S3 
0.iO 


\m 

.26 


Respondent reporting 
for other famliy 
memberi 








Chronio and asute 

conditfoni ^ . . . . 
Physioian visits . . , . 


i.eo 

0.14 


1.2S 
0.26 


A2 



V ^ .01. 

performmce as they gained experience, at least 
not during a single study! In fact, performmce 
began to deteriorate almost immediately after 
training andj at least in some cases, dropped 
iignificantly within a few weeks, Contraiy to 
eKpectations, this deterioration occurred among^^ 
both experienced and inexperienced inter- 
viewers. 

The fragmenta^ data are susceptible to vari- 
ous inteipretations. One explanation is that 
reporting bias is related to the interviewer's 
inability or lack of incentive to encourage 
enthusiastic performance by the respondent and 
that, over time, intemewers become less con- 
scientious in their use of intemewing tech- 
niques. Because performance dropped as the 
intemewng period progressed, it would seem 
tharperformaiice is related more to motivational 
level than to ability to perform the t^k* Even 
though the nature of the beha^doral change in 
the intemewer is not known, there is evidence 
that the manner in which, the questions are 
'aiked^ may change over time. Whatever the 
mechanisms are, the interviewer's behavior 
apparently has an effect on the respondent's 
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beha\7*or, both-=in the reporting during the 
Mntem^irw itself and in the respondent*! subse- 
quent performance in filling out and mailing a 
ielf-admihiitered form* 

It iSj of course, no surprise that lack of 
incentive or interest in performL.g the intemew- 
ing task results in poor performance, but it is 
surprising that the interviewer*! performance 
■ dropped as rapidly as these findings indicate* 

It appeiyrs that interviewers were being 
" reinforced for part of their role performance, 
and that the reinforcement brought about 
improvement. However, adequate performance 
in stimtdating respondent reporting behavior and 
in proper use of interviewing techniques was not 
reinforced and, accordingly, deteriorated* Per- 
formance in interviewing was not reinforced 
because observation of interviewers was not 
continued after training, and good- or poor 
performMice could not be identified by a review 
of the completed questionnaire* (It is instructive 
to note that in one of the studies reported here 
the interviewer who was among the poorest in 
obtaining known hospitalizations and whose 
underreporting rate rose sharply over the 
interviewing period was selected ai the best 
interne wer and vim promoted to m office 
supervisory job becauie of her "eKcellent" 
work.) 

There are two m^or implications in these 
findings. The first is that research is needed to 
identify explicitly those interviewer behaviors 
that relate to adequate reporting* Studies that 
concentrate on interviewer-respondent interac- 
tion, especUly through ansJysis of recorded 
intemews, can provide information for better 



understanding of some of these factors. VA; 
second implication is the need for training and 
supervisory methods that will reinforce adequate 
interviewer performance in the application of 
interviewing techniques* 

Studies frequently show a relationship be- 
tween the information reported in interviews 
and the attitude ofj or expectations of, the 
interviewer* Interviewer bias studies abound, but ; 
all relate characteristics of the interviewer 
(backp*ound characteristics* attitudes, percep- 
tions, and expectations) to the reporting of 
respondents. The data presented here point to 
the importance of the intemewer*s continued 
motivation to do a conscientious job of 
intervievring. The iuggestion is that as fieldwork 
progresies, intemewers become less airef ul or 
conscientious in using the techniques they were 
trained to use* Furthermore, there is evidence 
that interviewers who performed well in the 
interview also inspired their respondents to 
perform well, as was shown in the adequacy of 
reipondent reporting of hospit^zations in a 
sel^adminiitrative procedure. Good interaction^ 
improves not only the technical ^^pects of the ' 
interview j but also stimulates a high level of 
respondent role perform wice which extends in 
time beyond the interaction itself. 

Interviewer training needs to include devices 
for building the interviewer*! enthusiasm for the 
job as well as procedures for training in the ; 
proper use of techniques: Attention to perfor- 
mance cannot stop when interviewer training is 
completed. Training needs to be evaluated and 
good performance reinforced continually during ; 
the period of production interviewing* 



THE USE OF VERBAL REINFORCEIVIENT 
IN INTERVIEWS AND ITS DATA ACCURACY 



The purpose of this section is to describe 
interne wer verbal reinforcement as it occurs in 
intemews,^ to conceptualize it in the same 
mmner as other major interviewer behaviors, 
and to look at some of the relevant research 
studies that provide information about how 
verbal reinforcement influences the accuracy of 
interview data. 

The major category of interviewer behavior of 
concern here is interviewer feedback* Inter- 



viewer feedback consists of evaluation state- 
ments that the interviewer mrices after the 
respondent says something* The subject of feed- 
back may be approached through two basic 
questions* "How can interviewer feedback be 
used to improve data?" and "How can it be 
arranged so as not to introduce unwanted bias?" 
The following discussion will be limited to 
interviewer verbal reinforcing statemmts which 
represent only one kind of interviewer feedback. 
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.Probes and statements that are intended to build 
i rapport are not treated directly* While this 
restriction is undesirable/ it ^^i^ by 
the ^act that research conducted at the Suwy 
-.^Research Center and elsewhere has dealt mainly 
; with task-oriented verbal reinforcing statements/ 

RESEARCH ON INTERVIEWER 
t ; REINFORCEMENT 



J 5ince Pavlov's fainous experiments, there have 
been numerous studies of the effects of rew^d 
and reinforcement on learning and performaice/ 
Much of this research has been carried out with, 
animds and occasionally with children. It has 
been assumed that the principles of reinforce- 
ment derived from these laboratory studies 
could be prcyected to adult human beings. This 
assumption has been made explicitly by many 

writers, most notably Dollard and Miller^ i and 
Skinncr.5 2 

Operant Conditioning Studies With 
Virbar RtinforeemCTt 

. Outside^ the psycholo^ laboratory two 
Important kinds of practices, programed instruc- 
tion (or teaching machines) and behavior modifi- 
cation therapy^ demonstrate the powerful ef- 
fects of properly scheduled reinforcement on 
human perform^ee. Some practitioners in the 
fields of education and psychotherapy have felt 
that traditional methods of producing learning 
or beha\dor change (e.g,, lecture, introspection) 
^5 ^'^i^ffi^J®'^** They have met some success with 
v J^^'^iqu^^ based on tightly controlled 
Dperant conditioning techniques where re- 
inforcement is contingent on the respondent's 
ictual behavior. 

These studies point to the importance of 
:eedback on humm performance under a variety 
jf conditions. However, the experimental situa= 
ions mentioned above (sentence construction, 
ree association, and casual conversation) are not 
ike a typical^ survey interview* Sweral studies 
lave been conducted in interview settings; The 
irst set of studies shows that interviewer re- 
riforcement has great effects on the amount of 
eipondent behavior. The second group of stud- 
;s shows that interviewer reinforcement can 
roduce "an interviewer bias effect. Finally, the 
tudy by Marquis, Cannell, and Laurent^ ^ indi- 



cates that, under certain conditions, interviewer 
feedback can be used to iricreaje the accuracy of 
information given by reipondentil : 

iffi^ of Faedbiok on Amount Repdrl^ 

A study by Marquis and CannellW ihows that 
the presence or absence of intemewer verbal 
feedback has a sipiiflcant effect on the amount 
of health information reported by a respondent 
in a household interview about family health. In 
this study a probability sample of moderate- 
income females between the ages of 1 7 arid 65 
and Imng within the city limits of Detroit was 
interviewed. They ; were asked quMtions a^out 
their own health, about their use of different > 
kinds of medical services^ and about the health 
of another person in the household. Although 
three experimental groups were used in this 
study j two groups of reipondents are of interest 
here; (a) the group of respondents who were 
reinforced every time they reported a symptom 
or health tondition either for themselves or for 
the other person in the. household for .whom 
they,^were^reportingr and -(b)- the'-^grdu^^ 6^^ 
respondents who were interviewed using the 
same questions but who received no reinforce- 
ment orfeedback from the interviewer/ 

^^P^^™*^'*t^l poup the kinds of 
reinforcing statements which the ^ interviewer 
used are outlined here. Words in parentheses 
could be used, omitted, or interchanged by the 
interviewers . . ^ v ; ; ; 

(Yes.) That's the kind of Information we need; ^ 

(Um-hmm J We're Intirestid in tha^^^ 

(IseeJ You have (name of condition) 



There were some other small cUfferences in the 
intemewing procedures for the two groups and 
these are described in detail elsewherel^®^ 



-The thifi! poup wis dio intemcwad without wmforcc- 
mmu A list of iymptoms, which appcmd at the biginning of 
the other two kindi of int^'ewi, \m msmtd ne^ the end of 
the third fr^tment int^ew. Ite pu^ose of this pfoceoiir* 
to Jest the sensiMng afftcy of the symptoms list on 1^ 
f^orting^ No main ef^ %vas found on the am^nt F^orted 
iii the int^ew, althou^ It i^y be necessiry to be^n a 
reinforcemsit interview wiA a iymptoms list which mown aU 
fespondehts to report some tort of siekneii aad to receivr 
reinforeemCTt for their f porting* - - 
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The main dependent variable in the study was 
the number of chronic and acute conditions 
which the respondent reported for herielf. 
Following the logic outlined by Cannell, Fowler, 
and Marquis*^ (appendix II), it was assumed 
that the more chronic and acute conditions 
reported, the more valid the ^overall data. The 
group of respondents receiving reinforcement 
reported aai average of 2.74 conditions per 
persbn* The nonreinforced group reported an 
average of 2,20 conditions. The data indicate 
that the reinfdfce"d"^^^pup of respondents 
reported about 26 percent more chronic and 
acute conditions than did the nonreinforced 
group. The difference is significant at the .02 
level of confidence. The same magnitude of 
effect was obtained for reporting chronic and 
acute conditions for the other person in the 
household. The number of conditions reported 
by proxy averaged 1.88 in the experimental 
interview and 1.43 in the control intemew. The 
experimental or reinforcement interview 
obtained about 24 percent more chronic and 
acute conditions reported by the proxy 
reipohdehf W 

the .01 level of confidence; Other reiults 
indicated that the reinforcement effect was 
general rather than limited to one kind of 
condition. For example, the reinforced respond- 
ents reported about 25 percent more medically 
attended conditions for themselves and about 55 
percent more of the highly embarrassing 
symptoms than did the nonreinforced group. 
The group that received feedback from the 
interviewer did not report^ more visits to 
physicians than did the nonreinforced group. 
This finding is .discussed in more detail in the 
original report by Marquis and CannelL^®- 

Kanfer and McBrearty^^ interviewed 32 
female undergraduates about 4 predetermined 
topics. During the first part of the interview the* 
women were haiided cards listing each of the 
topics and were asked to talk about each one. 
The experimenter sat in the room but kept 
completely silent. During the second part of the 
session the experimenter reinforced the respond- 
ent whenever she tUked about a predetermined 
two of the four possible topics. Reinforcement 
consisted of a "posture of interest" including 
smiles and the phrases "1 see^" "Um-hmm/Vand 
**Y€S.*' When the women talked about the 



remsdning two topics7 the experimenter said and 
did nothing. The subjects spent more time 
talking about the topics that were reinforced 
than about those topics that were not 
reinforced. 

These two studies indicate that interviewer 
feedback can have a great effect on how much is 
said during the questioning. It should be noted 
that both of these studies involved maximum 
contrasts in interviewer feedback; that is, in one 
group or at one time interviewers used verbal 
reinforcement md for another group or at 
another time interviewer feedback was com- 
pletely nonexistent. While these studies show 
that interviewer feedback can have important 
effects on the amount reported, they indicate 
little about the nature of the effects or what 
kinds of feedback procedures are most effective. 

Use of Ftedbiok to Increase Accuracy 

The study to be discussed here is one in which 
the data indicate that interviewer feedback 
procedures can have beneficial effects on survey 
data accuracy. It should be ^ remembered that^ 
this is a single study dealing with fact responses, 
and that the results are tentative and possibly of 
limited generalization. This study by Marquis, 
Cannell, and Laurent^^ was carried out under 
contract with The National Center for Health 
Statistics. In an unpublished study by Marquis 
and Laurent,^* a number of interviewing 
variables were tested but the discussion that 
follows is restricted to the effects of reinforce- 
ment on initial interviews and reinterviews using' 
short (nonredundant) questions. The sample 
respondents were white females between the 
ages of 17 and 65, who were residents of the 
greater metropolitan Detroit area. The respond- 
ents were selected on a weighted basis from 
persons who had come into a prepaid health 
clinic during a 6-month period prior to 
interviewingv During the time each patient was 
in the clinic a physician filled out a checklist 
indicating which of 13 chronic conditions the 
patient definitely or probably had, and about 
which of the checklist conditions the physician 
had no information. The physician obtained 
information about the chronic condition from 
the patient's: record and from his own 
knowledge of the patient's medical condition. 



. He did not interview the patient directly about 

:- the spedfic conditions listed o 

The dependent variable used in the study was 

;p the accuracy with which respondents, m a series 
of subsequent household interviews, reported 
having had or not having had each of the 13 

■ aforementioned chronic conditions. Two types 
of respondent error were (a) underreporting, in 
which case the respondent failed to mention a 
condition which the doctor said she had; and (b) 
overreporting, where the respondent mentioned 
a condition which the doctor said she did not 
have, ■ ^ 

It was^^not expected that the respondent and 
physician would agree entirely on the state of 
the respondent health* However, what was 
cKpected, for the reporting of these 13 very 
common chronic conditions, was that an 
experimentaL interviewing procedure that ob- 
tained low average mtes of respondent-physician 
disagreement obtained more valid data than an 
interview procedure yielding high rates of 
disagreement. The study included a reinforced 
group of respondents and a nonreinforced 
groupf All respondents received the same kind of 
.questionnaire composed of straightforward short 
questions. The verbal reinforcement used in the 
remforced group was similar to that described in 
connection wth the study by Marquis ^d 
CannelL^^ Half vof the respondents were 
reinterviewed approximately 2 weeks after the 
initial interview. Respondents in the experi- 
mental group received reinforcement for report- 
ing all kinds of health events, not just symptoms 
ind conditions^ as in the 1971 study just cited/ 
The results obtdned in the analysis of the 1972 
study data were not those initially antici- 
pated.^^: They indicated that ^he efhcts of 
different ways of eonducting inter\iews were 
mediated by the level of education of the 
respondent. Procedures that increased accuracy 
and reduced bias in the low education group 
(had not completed the 12 th grade) had 
opposite effects in the high education group, 
rhe reader should consult the full report of this 
itudy^ ^r a more detailed treatment of all data 
md all experimental techniques. 



The discusiion of the results of this study is 
limited to four groups of respondents, those 
^ remforced and those not reinforced in eac^^ of 
f*^^ education groups, all of whom were 
mterviewed with short questions * 

The underreporting index sco res showi . in 
table 37 indicate that, for respondents who had 
not finished high school, the use of 
reinforcement in interviews significantly reduced 
the underreporting of chronic conditions in the 
oripnal interview. However, reinforcement 
significantly increased underreporting en-or for 
the more educated group. The drop from .387 
to .225 in underreporting brought about' by 
reinforcement in the less educated poup repre- 
sents a decrease of about 42 percent i the 
percentage incre^e of underreporting in the 
higher education group due to reinforcement 
(from .443 to .575) was approximately 30 
percent. Both of these differences are sipiificant 
at the 5-percent level. 

The overreporting errors were generally low 
(7 to 14 percent) and were less affected by 
.. reinforcem€nt..,The . slight ^ trends in the- data- 
suggest that reinforcement reduced over- 
reporting in both education p-oups. One hypoth- 
esis in this itudy was that reinforcement by the 
mterviewer each time the resporident reported 
an illness would result in the resp onden t "invent- 
ing" sicknesses in order to gain further approval 
of the intendewer; Since^ the reinforcement^ 
mterview produced fewer felse positive 
responses than the intemew that did notlndude 
reinforcement, the trend was counter ^o tiiat^ 
expected. 

The total error index scores shown in table 37 
represent the unweighted sum of the jndividua 
imderreporting and overreporting scores, Sinie 
undeireporting scores had a hi^er mean and 
variance than overreporting scores, the total 
error score reflects the effects of underreporting 
due to reinforcement and education level. For 
the less educated group, reinforcement sipiif^^ 



^^Copies of thif study are availible from the Natjonml Center 
if Health Statistics, Viial and Health Statisiics, fm Pub. 
o. 1000-Serici 45. 



The reinforcement Gontmt ^sq tested using a 

queiUDnnaire ^*ich contab a mixture of long and short 
questiDni. For reasoos not discuised h«, the effect of 
reinforcement t^th long questions wis ambi^DUs. It is cJm, 
however, that the comblnaUon of rdnforcem«t with long 
quesUons does not result In marked improvement In ' ita*'"^' 
accm^cy. --. . / r. 
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Table 37« irror ratti fef chronic condition riporting in Original Initrvltm an6 peintarvltwi, by use of ralnfarctmt nt end tducatlon 

: l©val of respondint , r 





Original interview 


Reinterview 


* Respondent's educatioifllevel 

.j?. .... ..... . . ■ - . .. 

... ^ ■ ... - ... . ; , ■ . - 


Reinforced 


Not 
reinfor^d 


Reinforced 


Not 
reinforcod 


--^vr*- ... . . . 

' - : : . Low education level 










(Murnbif of rtsponden^ . . . . ^ * * ^ ^ ^ . , . . ^ . . . . . . 


iO 


41 


2i 


20 




.33a 


.530 


.381 


.§66 




.225 
.113 


.387 
.143 


.230 
.131 


.400 
J66 


High education level 












38 


53 


20 


27 




,i44 


*534 


.603 


.61 § 




.069 


.443 
.09! 


.§21 
.082 


. .4i§ 
.120 



cmdy v-^ efrrd tha total error; for the more 
educated gioup, rdnforcement significantly in- 
creased the overall error. 

The trends in the reintemew data were in 
general iimilar to those observed in the data as 
collected originally. The questions were approxi- 
mately the same as those asked the first time. 
Respondents who were in the reinforced group 
originally were also reinforced during the re- 
intemew. Similarly^ respondents initially in the 
nonrein forced group J did not receive reinforce- 
ment the second time. Thus, the reinterview 
data reflect the- cumulative effects of either 
reinforcement or nonreinforcement. 

While the results of this study are not as clear 
as one might wish? they do indicate that certain 
kinds of reinforcement procedures can have a 
marked effect on the accuracy of the estimates 
made for a population from survey data. It 
would appear that less educated respondents 
rely on interviewer cues to direct their reporting 
performance, Thus^ appropriate use of reinforce- 
mentj probes, and odier feedback by the inter- 
^ewer can aid recall ^d reporting accuracy. 
Perhaps because of the perceived social status 
discrepancy J feedback is accepted as appropriate 
and actually welcomed. Performance is generdly 
good in an appropriately conducted initial inter- 



\dewj but there is little additional benefit from a 
reinterview. . . • • 

More educated respondents carry out the 
reporting task according to their own under- 
standing of it rather than relying on cues from 
the interviewer. Reinforcement under such cir- 
cumstances may be perceived by the respondent 
as inappropriate^ unnecessary, or even con- 
descending. The more educated respondent 
appears to have a tendency to underreport 
chronic conditions, possibly due to a stronger 
conservatism or social desirability bias. While 
these findings are somewhat speculative, they do 
stress the importance of such * "human'* charac- 
teristics as memory, recall, cognition, social 
status, md intellectual ability, that are often 
overiooked in methodological studies. 

The studies cited to this point indicate that: 

a. Intemewer feedback makes a difference in 

respondent performance, 
b- Intemewer feedback can bias respondent 

answers. 

c- Intemewer feedback can increase response 
validity, . 

However, these findings are based on experi= 
mental interview studies with unorthodox ways 



of programing faedback and do not indicate Kow 
feedback is used in the normal household 
interview. 

Use of FeedbaGk in a Nonixperimentel 
Interview 

Marquis and CmntiP obtained tape record- 
ings of 181 interviews with employed male 
respondents about employment-related topics. 
The respondents included both black and white 
persons ranging in age from 18 to 64. The four 
interviewers were white middle-class female 
residents of suburban Detroit. 

The tape recordinp of each interview were 
analyzed by identifying each instance of verbal 
behavior on the part of the interviewer or the 
respondent and assigning it 1 of 36 possible 
behavior codes. Thus^ each interview was trans- 
formed into a series of code symbols which 
represented the kinds of behavior in which the 
intemewer and respondent, en^ged. The data 
indicated that 23 percent of all coded inter- 
y^dewer behavior consisted of feedback. Feed- 
back, as a^ category of interviewer behaviors 
occurred as frequently as probing. The only kind 
of interviewer behavior that occunred more 
frequently than feedback was question-asking 
(this accounted for about 37 percent of aU 
interviewer behavior). These data indicate that 
intendewer feedback isj indeed, a large part of 
interviewer behavior,^ 

: Data presented in table 29 are the basis of 
discussion of the use of reinforcement by 
interviewers. The table shows a very surprising 
findingr The probability of feedback after an 
inadequate answer was almost the same as the 
probability of feedback after an adequate 
answer. Even more surprising is that interviewers 
on .the average -reinforced : over half of the 
respondents- refusds tor answer, ,Tlu^ 
pattern discloses that interviewer feedback may 
be used in an inappropriate manner. These data 
indicate that interviewers not only gave positive 



As . this fcpon. goes to ^ press, T^ulti from a imaU pflot 
sfecrvitlon study (N-2S), conducted on a different kind of 
jitemew (urban problems) and with more experienced int^ 
dewerSj Indicate that the proportion of mter^ew^ behavior 
ifvoted to shorti poiitive feedback statem^ti was 10 to 15 
^ercent^ Hiat this percentage fan^ is considerably lower thMi 
he proportion obtained In the labor for^ study suggests the 
leed for furtho^ research. 



evaluation- responses when the respondent re- 
plied adequately, but they also said positive 
things in situations where they may have been 
under some tension or may have felt the need to 
build rapport in the interview,: The rapport 
hypothesis about the use of feedback is sup- 
ported by the finding that feedback was given 
after refusals to answer and after answers that 
did not meet the objectives of the question. 

The authors have never syitematically dis- 
cussed these data with interviewers, but were 
this to be done, it might be expected that 
intervieweri would insist that the difficulty of 
their job makes it necessary for them to build 
rapport with respondents by using positive 
reinforcing statements when tension is felt 
during the- intendew* The authors would prob- 
ably reply that at least one experimental study 
h^ shown that validity can be improved by 
reinforcing only adequate answers and that this 
pattern should be followed. Possibly these two 
divergent hypotheses about the effective use of 
intendewcr reinforcing statements cm be tested 
experimentally. At this point there is only 
limited empirical support for the authors* posi- 
tion and intuitive, common-sense support for 
the interviewers' position. 

Dasou^lon ^ 

The data indicate that the survey researcher 
should be concerned about the feedback styles 
used by intense wers. The remainder of this 
section discusses ideas and reseu'ch concerned 
with reinforcement effects in the personal inter- 
view* Before going further into the hypothesized 
effects of verbd reinforcement on respondent 
performance, a schematic : diagram of variables 
that may influence reporting quality might be 
helpful* ^ ^ 
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KNOWLEDOE 
OF WHAT IS 
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RESPONDfNT 
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RESPONDING 
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REfifONOENT 
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The diagram impliei that there are four hypo- 
thetical personal characteristics (other than hav- 
ing correct answers available in memory) that 
will affect the accuracy and completeness of 
reporting. These consist of two khowledge (cog- 
nitive)' variables— knowledge of what is expected 
and knowledge of how well one's responses are 
meeting these expectations— a skill variable, and 
a motivational variable. 

This diagram implies that if the respondent 
knows what is expected of him, has the ability 
to do vyhat is expected^ can tell how adequate 
his responses are vis a vis the task requirements, 
and wants to do well, then the data he gives will 
be accurate and complete. The purpose of the 
discussion that follows is to explore how verbal 
reinforcement procedures and alternative proce- 
dures affect each of these hypothetical variables. 

The tentative conclusion reached is that 
positive verbal reinforcement contingent on 
adequate answers provides a wide variety of 
desirable effects on these intervening constructs, 
while other procedures that might be used tend 
to have more limited effects. The effects of 
verbal reinforcement at any particular stage arc 
not totally clear and derivations from theory 
suggest reinforcement may also have some 
counterproductive effects in certain situations. 

THREE KINDS OF INTiRVIEWER 
VERBAL REINFORCEMENT EFFECTS 

The effects of verbal reinforcement on 
respondents can be divided into three categories* 
a cognitive effect, a conditioning effect, and a 
motivational effect. There can be a great deal of 
overlap among these categories but, for ease of 
presentation, the three kinds of effects will- be 
treated as if they are independent. 

Interviewer verbal reinforcement has a 
cognitive effect on respondents because it 
supplies information about the expectations of 
the interviewer and how adequately the 
respondent is meeting these expectations by his 
answering be havi on For example, if the 
interviewer asks whether the respondent has ever 
had a headache, the respondent says Yes," and 
the interviewer uses a reinforcing statement such 
aSj "Good, that's the kind of information we 
need/' The interviewer has indirectly told the 
respondent that she expects him to report his 



minor illnesses and that the response he just gave 
wa ?'Qod one in terms of meeting the 
obj. of the health interview* This 

inforiiiauan-giving aspect of verbal reinforcing 
statements which changes (or maintains) the 
respondent's intellectual understanding of the 
respondent role requirements and the adequacy 
of his responses is described as a cognitive effect. 
The second effect that interviewer feedback 
may have is referred to here as "instrumental 
conditioning." This effect is important in many 
studies of the psychology of learning. In the 
simplified model of the interview just described, 
the interviewer's evaluation comes immediately 
after the respondent ha.s given a response* This 
sort of pairing of an evaluation with a response 
can have reinfqrcing properties, that is, it can 
alter the frequency of the b^^havior that 
immediately preceded it* Thus, the evaluation 
process has the potential of strengthening or 
; weakening the probability of eliciting that 
behavior or similar behavior on subsequent 
trials. Also, through the process of differential 
reinforcement of successive approximations^ the 
reinforcement procedure can establish a new 
kind of response class or behavior in which the 
respondent would not normdly engage on his 
own. 

The third possible effect of inter\dewerj 
feedback is motivational* In this case feedback 
affects the intensity or psychological effort 
which the respondent alloeates to his reporting^ 
task and to other behavior that may interfere 
with the adequate performance of his 
respondent task. 

Cognitive Effects of Reinforcement 

Two kinds of knowing or understanding are 
thought to influence respondent performatice: 
understanding ,what is expected (e*g., knowledge 
of the proper respondent role) and knowing 
when answers meet and do not meet those 
expectations. Two ideas are hypothesized: (a) 
such knowledge is sometimes very helpful but is 
neither necessary nor sufficient for good 
perforraance in some conditions; and (b) 
reinforcement plays an important role in helping 
the respondent to acquire both kinds of 
knowledge and is a more effective procedure 
when compared to more conventional teaching 
techniques. = 



Before exploring the effects of reinforcement 
on knowledge, it should be noted that many 
people mmntain that the most effective way to 
increase a person's knowledge and understanding 
is to teach him by direct (e^g., to lecture) rather 
than by indirect (e.g., using feedback) methods. 
Direct approaches to teaching which are not 
dependent on respondent perform^ce. are^^not 
uncommon in survey interview settingSp For 
example, before an interviewer arrives at the 
doorstep, the potential respondent has often 
received a letter or a brochure that explains the 
survey and describes what is wanted from the 
respondent. Often at the beginning of the 
interview, an explanation of the goals of the 
research is given, accompanied by specific 
appeals for accuracyj completeness, or candor, 
_ ^ ^.VeEy„ Uttla is knawn^ about the effects- of this - 
commons direct approach to teaching a 
respondent. However, if it were possible to 
extrapolate from the lecture analogy and to 
research these effects, it might be found that 
often the "students" had not learned or 
understood or would not be able to verbalize 
what they had been expected to learn* Two 
studies obtained some relevant data* In a study 
by Cannell, Fowler, and Marquis* § respondents 
were reinterviewed 1 to 3 days after an initial 
health interview. Two questions were asked of 
412 respondents to find out how well they had 
understood their role. The answers were 
distributed as follows: 



Q, 26. Did the interviewer want you to be exact in 
the answers you gave or were general ideas 
good enough? 





Percsnt 


Respondent's answer 


distrN 




butfon 


"f xaat" , 


Si 


"Some of etch' V 
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3B 
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. Q. 27* Did she (interviewer) want everything^ no 
matter how small it was^ or was she interested 
only in fairly important things? 





Percent 


Reipondent's answmr 


distrl^ 




butlon 


"ivirything" 


76 


"Important things" 


19 


Not aicertijnf d 


i 



A similar question was asked of 428 reipondents 
in another study by Marquis and Cannell* 0 at 
the end of the health interview rather than at a 
followup interview: 



Q. 17* Will people think we want them to rq>ort all 
their illneisis or only the important ones? 
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The data indicate that despite introductory 
letters and brochurei, inter\dewer eKplanationis 
and the actual experience of the interview, 
between 20 and 40 percent of respondents in 
these health studies clearly did not understand 
the respondent role correctly. 

Some data are m^ailable to ii^dicate what 
techniques are effective in changing respondent 
understanding. In an unpublished Survey Re- 
search Center study by Cannell/ Fowler, and 
Marquis,^ 6 different kinds of brochures and 
introductions were used. The effect on respond- 
ent understanding of an unattractive but 
informative (control) brochure was compared 
with that of: 

a, A brochure indicating the kinds of 
questions to be asked, the interviewer*S: 
role* the respondent's role^ and the 
importance of accuracy in reporting; 

b. A brochure mentioning uses of data and 
stressing the benefits that might result from 
the survey; 



These reasons indicmtcd that reporting ^ illness wu what 
the survey, intervlevyeTi or Government wanted or that reporting 
all illneises would result in good data« - 



c, A calendar on which the respondent might 
write family lickness information for two 
weeks prior to the health intervdew; and 
^ d. A set of actual question! to be asked during 
the interviews asking the respondent to 
think about them and consult records and 
family members for accurate information* 

Results of pretests were so disappointing that 
full scale evaluations of the effects of these 
communication/teaching devices were not 
undertsiken. The experimental materials, 
whether used singly or in combinations made no 
difference in respondent knowledge or reporting 
performmce. 

It is currently believed that teaching may be 
more effective if methods of programed 
instruction are used. In contrast to the brochure 
or lecture t€chniquW,"ohe essen 
the method of programed instruction is 
immediate feedback about the adequacy of each 
answer the sti^ent gives. One characteristic of 
reinforcement is that it can provide immediate 
information to the respondent about the 
adequacy of his answer. The first SRC 
reinforcement study rnentioned above^^ gave 
positive verbal reinforcement for adequate 
answers (see report of that study for feedback 
procedures a^d definition of adequate answers) 
to one group and not to another group. Both 
^oups received the same advance letter and 
explanation of the study by the interviewer. The 
following data indicate that the reinforced group 
had a slightly better understanding of the 
respondent's task than did the group not 
receiving feedback for adequate answers: 



Q* 17. Will people think we want them to report all 
their illnesses or only the important ones? 
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Therefore, a reinforcement procedure does 
have at least a smsdl potential for teaching a 
respondent. However, reinforcement effects 
appear to be stronger in the area of eliciting 
adequate responses than in teaching what is 
expected, since the effects of reinforcement on 
the actual amount reported are greater than the 
effects on understanding the task requirements. 
Two remaning points about verbal 
reinforcement and knowledge effects on 
reporting performmce need to be discussed: (a) 
certain types of reinforcement procedures used 
in the laboratory experiments mentioned above 
may be less effective than the procedures used in 
the 1967 study; and (b) the effects of the verbal 
reinforcement procedures used in the 1967 
study did not appear to be mediated by the 
dep^ee_ .of __ the .„ respondent 's_ kn o wledge^ ._o f ^ 
expectancies (awareness of respondent task 
requirements). 

There is a lively controversy over whether or 
not the respondent must know what is expected 
of him in order to perform well* In other words, 
is complete awareness of what is expected a 
necessary, although possibly insufficient, 
condition for accurate reporting? Despite the 
finding that reinforcement appeared to increase 
understanding of the response requirements, the 
data from the 1967 study mentioned above 
show no correlation between reporting and 
awareness as measured by question 17. This is 
true for the reinforced group as well as for the 
nonreinforced group. Yet the reinforced group 
showed superior performance in reporting their 
health conditions. Cannell, Fowler, and Marquis 
reported a slightly positive, but not statistically 
significant, relationship between awareness and 
good reporting performance,*^ Fowler 
reanalyEed the latter data and found that the 
relationship between awareness (measured by 
one of the two questions) and the amount of 
information reported was large for respondents 
with^ at least a high school education.* * That 
awareness, however, did not predict reporting 
behavior for respondents with less education, 
nor did data from the second awareness question 
predict behavior. Possibly the reinforcement 
effect, if it is cognitively mediated, is produced 
by letting the respondent know when his 
responses are adequate rather than by giving him 
knowledge of general task requirements. This 



4i 



implies that the respondent need not understand 
exactly what is expected of him to respond welL 
Recent experiments in social psychology have 
tried to explore the relative effects of awareness 
of task requirements and reinforcement of 
performance. A detailed treatment of these 
findings is beyond the scope of this report, but 
some of the main issues arid findings are 
presented at the end of this section where 
further experimentation is discussed. It appears 
that awareness can be helpful when the 
respondent (a) can tell when his single responses 
are meeting task requirements, (b) has the skill 
to perform what is requested, and (c) has the 
will (motivation) to perform* 

Feedback used in the 1967 studies may have 
provided two types of information, and achieved 
the increase in reporting frequencies because of 
this double-barreled effect; In some of the 
laboratory experiments mentioned earlier, re- 
inforcing statements contingent on correct 
answers consisted of *'Yes/* "Mm-hmmp" 
"O.K.,'* or "Good,** The 1967 studies used 
verbal reinforcements which contained more in- 
formation, statements such as: "Mm-hmm, we 
need to know that,** or "I see, that*s the kind of 
information we need.** Probably the two types 
of statement convey different amounts and 
kinds of information. The simple statements 
apparently convey information about response 
adequacy, leaving it to the respondent to infer 
the interviewer*s goals. The longer statements 
possibly make it easier for the respondent to 
arrive at some knowledge of interviewer expecta- 
tions* This hypothesis would be relatively easy 
to test in the laboratory or in field interviews. 

These conceptions of how reinforcement 
teaches the respondent to understand his proper 
role point to the desirability of using more effi- 
cient techniques which do not rely so heavily on 
the respondent's ability to figure out what is 
expected. Further research seems to be needed 
in order to test whether the respondent's under- 
standing of his role is necessary for good report- 
ing or whether knowledge of the adequacy of his 
responses is, in itself, sufficient. If knowledge of 
expectations is found to be important, it might 
be effective to develop a method that informs 
the respondent of his expected performance 
quality and at the same time, through reinforce- 
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ment procedures, gives him immediate informa- 
tion about the adequacy of his answers. 

The effects of reinforcement on two kinds of 
respondent knowledge and the role of knowl- 
edge in producing good survey data have bein 
touched upon above. Other variables can also 
influence data quality. In the following sections 
reinforcement is discussed in the context of two 
other variables, skill level and motivation. 

Conditioning Efftati 
of ReinforcemOTt 

Experiments cited earlier show that giving re- 
inforcement immediately after some behavior in- 
creases the probability that such behavior will 
recur. This pattern is defined here as an operant 
conditioning effect, or more simply a condition- 
ing effect. In the tradition of B. F. Skinner, the 
arrarigernent of reinforcement contingencies to 
produce the conditioning effect minimizes con- 
sideration of the intervening variables that might 
help explain the obtained results. Thus, the fol- 
lowing paragraphs omit speculation about how 
interviewer reinforcement may affect cognitions : 
and motivations to achieve a performance effect* 
They relate to such problems as identifying the 
correct behavior to reinforce and the circum- 
stances under which a conditioning procedure 
seems appropriate/ 

When to use a condttiomng procedure,— The 
following discussion is based on the premise that 
conditioning effects of reinforcement should be 
used (a) when respondents do not have a clear 
understanding of how to perform effectively and 
cannot be given this understanding by mere ex* 
planation or appeals for good performance, (b) 
when the respondent does not possess the ability 
or skill for good reporting regardless of whether 
or not he understands what is wanted, or (c) 
when the respondent is performing in a manner 
which is less than optimal (for example, he 
wants to do something besides answer questions 
accurately and completely). The implication is 
that if the respondent understands what he is 
supposed to do, has the ability to do it, knows 
when he is doing it, and wants to do it, there is 
no reason to introduce a conditioning procedure 
into the interview. However, all of these condi- 
tions are seldom met in any personal intemew : 
situation* 



The general conditioning pattern suggested 
for personal interviews is the scheduling of re« 
in forcing statements only after adequate 
answeri. In this way the frequency of adequate 
answers can be expected to increase. For exam- 
ple, an adequate answer appears fairly easy to 
elicit with increasing frequency when the follow- 
ing instruction is given: "When I ask an open- 
ended question^ please tell me all you can think 
of, using specific examples*" This is accom- 
plished by an interviewer using a reinforcing 
statement every time the respondent gives an 
acceptable idea or example in response to an 
open-ended question. The arrangements to pro- 
duce an increase in valid answers, especially to 
forced-choice questions^ are not that easily con- 
ceptualized* Methods that change the reinforce- 
ment contingency rule^ to reward responses that^ 
come closer and closer to approximating some 
goal have been shown to be effective and, 
theoretically, could be used in the personal 
interview setting* 

Another use of conditioning procedures is to 
maintain an acceptable level of respondent per- 
formance. Once the respondent has been taught 
appropriate response behavior, it is desirable for 
him to continue to answer acceptably* Accord- 
ing to the operant conditioning literatures a pre- 
viously reinforced response (especially if it has 
been reinforced every time it has occurred) will 
be given less and less frequently if it is no longer 
reinforced* Thus, another function of a verbal 
reinforcement procedure is to prevent extinction 
of appropriate behavior once it has been estab- 
lished* A schedule of reinforcement that uses 
positive statements after at least some of the 
desired behavior is probably adequate to serve 
the response maintenance function. Procedures 
that omit reinforcing statements or use them in 
a random way probably cannot maintain any 
performance pattern that has been established 
earlier. 

Problems of conditioning procedures for 
teaching.-- A conditioning effect of reinforce- 
ment involves the respondent learning how to 
perform or improve his performmce on some 
task* He may or may not be aware of his result- 
ing increase in ability. For a feedback technique 
to be effectives the interviewer (or study 
designer) must be aware of what is to be taught 



or improved, be able to recognize when a 
respondent performs the desired behavior, and 
be able to give appropriate verbal reinforcement 
immediately after the respondent behavior 
occurs. These are very stringent conditions, and 
in the usual interview situation they cannot be 
met* One major obstacle is the difficulty of 
determining immediately whether a particular 
respondent behavior is desirable or not. Without 
knowing which responses to reinforce, a proper 
conditioning effect cannot be obtained. 

The dangers of trying to teach one concept 
but actually teaching something else can be illus- 
trated by a recent series of experiments, Cannell 
and Marquis ori^nally diagnosed the problem of 
error in reporting "hronic and acute health con- 
ditions as an underreporting or ■■failure to re- 
port" problem. Subsequently, in their 1867- 
study,* ^ they reinforced every instance where 
the respondent reported a symptom or a chronic 
or acute health condition, and obtained a 25- 
percent increase in the number of symptoms and 
conditions reported* A subsequent look at un- 
published data about chronic condition report- 
ing from other sources suggeited that over- 
reporting might be more of a problem than 
underreporting for this kind of health informa- 
tion. Therefore, the conditioning procedure used 
may have decreased the accuracy of the data in 
that study because it led to increased overreport- 
ing errors, Thereforej Marquis, Cannell, and 
Laurent*^ carried out a second study using inde- 
pendent records from physicians as indicators of 
the presence or absence of chronic conditions 
for particular respondents* Respondents received 
approximately the same kind of reinforcement 
as in the earlier study. The results mentioned 
earlier in connection with the discussion of table 
42 did not show the expected biased condition- 
ing effect. If the conditioning effect were the 
only effect operating, it would be expected that 
overreporting (reporting of nonexistent chronic 
conditions) would increase. The data indicated 
just the opposite. The short-question reinforce- 
ment interview had its major effect in reducing 
overall reporting error by reducing the amount 
of overreporting* Exactly why tHirhappened is 
uncertain. Possibly no conditioning effect 
ocGurred, and the reduction in error was due to ■ 
increased knowledge or motivation effects. 



4nother possibility is that a concept other than 
a positive_or sickness-reporting response set was 
taught*^ The main point, however, is the diffi- 
culty in predicting what sort of consequences 
will arise when interviewers use reinforcement in 
a particular contingency schedule. While the 
above example shows that reinforcement proce- 
dures had a beneficial effect on the validity of 
health reporting, the effect may not have been 
for the reason originally hypothesized. 

Substitute for conditioning procedures when 
the problem is one of low respondent ability,^ 
Probably one of the biggest problems in survey 
interviewing concerns the respondent's ability to 
recall factual information accurately. The most 
dramatic example of a memory problem is 
shown by Canncll and Fowler^ in the reporting 
of visits to a physician during the 2. weeks pre--^ 
ceding the interview. Using a standard question 
designed to find out how mmiy times the 
respondent had consulted a physician during this 
timCj, 21 percent of the visits known to have 
occurred within the last week were not reported 
and 38 percent of the visits known to have 
occurred during the second week prior to the 
interview were not mentioned. Inability to 
remember increased over 80 percent from the 
first to the second week preceding the interview* 
It has been hypothesized that a conditioning 
procedure might help the respondent learn to 
iistinguish correct from incorrect memory rep- 
resentations of an event such as a physician visit/ 
In theory, this kind of teaching seems possible 
mt it has not yet been demonstrated as feasible 
n the inteiview setting. 

A nonreinforcement interviewing approach 
night be considered when ability to remember 



'^he most likely explanation for the data was how the re^ 
iforcemfnt variable was oripnmlly introduced to respondents, 
he interviewer began by asking the respondent if he had ever 
id any of each of 17 very common symptoms. The probabflity 
lat the respondent had experienced any one of these lymptomi 

least once In his lifetime was very high. Therefore, a "Yes" 
sswer was most likely a true answer and v^s reinforcedf and a 
^o'* answer was most likely an underreport and vm% not re^ 
forced. HiuSj the concept iiutially teught >^s probably, "Give 
ild answers and don't undeireport.." This concept may have 
ten maintained by the reinforcement schedule used in the rt- 
ainder of the Interview and was possibly still in effect for 
lestions about chronic oonditioni. : 



correctly presents problems for survey data 
accuracy. The recommended approach involveii 

a. Making the recdl task simpler so that mem- 
ory ability is less important for validity; 
and \ 

b. Using repeated trials so that an initially 
faulty recall decision may be corrected. 

Evidence for the possible efficacy of this task 
simplification and repeated trials approach is 
presented in another section of this report, 
"Memory and Information Retrieval in the 
Interview." It should be remembered, however, 
that if response error is thought to be due to 
variables other than memory failure, e*g., mis- 
understanding or fear of embarrassment, a re- 
inforcement procedure might be more appro- 
priate. _ ^ 

Motivational Efftrts 
of ReinforoamOTt 

The use of feedback statements, regardless of 
the rate or the contingency rule for their use^ 
may have motivational consequences. The moti- 
vational effects are probably not reflected 
directly in the awareness or consciousness of the 

respondent, and hence may be difficult to detect 
by questioning. 

One motivational consequence has been men- 
tioned previously. In standard interviews it was 
observed that feedback statements were used 
often in situations where there may have been 
tension or negative feelings, Iniplicit here is the 
hypothesis that positive statements by the inter- 
viewer can reduce the respondent's tension or 
hostile feelings about the interview. This 
hypothesis should be tested. 

Psychological theory suggests at least three 
possible motivational effects of positive feed- 
back statements. These statements might: 

a. Affect the general level of motivation drive; 

b. Strengthen or weaken levels of specific 
motives which facilitate or inhibit reporting 
performance; or ■ 

c* Affect the degree of approval the respond- 
ent has for the interviewer. The effect of 
approval on perforrnance is ambiguous and 
is discussed briefly below, . 
General motivation.— Some theories of moti- 
vation include the idea of "general drive" or gen- 
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eral arousaL This refers to a hypothetical con- 
cept about a nonipecific or general level of 
motivation which multiplies the strength of any 
behavior tendency in a person. For example, if a 
person is nmning a 100-yard dash, his speed will 
be faster if his general drive level is high and 
slower if his general drive level is low* His speed 
is affected by other things, of course^ such as his 
ability as a runner and his specific desires to win* 
It is hypothesized that reinforcing statements in 
the intemew increase the respondent's general 
motivation and thereby accentuate whatever 
response tendencies exist , regardless of whether 
the response tendencies are the right ones or 
not. 

A complete presentation of drive theory and 
supporting empirical evidence is beyond the 
. scope of this section^ but one fairly cpmplicated 
aspect of the empirical evidence is important 
enough to mention. (For further discussion, the 
reader should consult Zajonc* ^ ) There is a com- 
plex relationship between drive and behavior. If 
a task is easy, high levels of drive will result in 
good perform^ce/ If a task is difficult, high 
levels of drive interfere with efficient perfor- 
mance while low levels of drive are accompanied 
by good performmce* For tasks of intermediate 
difficulty, the level of drive shows a curvilinear 
relationship to good performance, with moder- 
ate levels of drive producing good results and 
high and low levels accompanied by poor perfor- 
mance. 

It may be that high levels of respondent drive 
reduce the underreporting problem in survey 
interviews and that this has been one effect of 
reinforcement procedures in the studies already 
discussed. There is some evidence that leads one 
to suspect, howeverj that respondent drive levels 
may have been too high in some of the experi- 
mental interviews. A schematic representation of 
' data from Marquis, Cannell, and Laurent^ ^ is 
given below. Possibly, the addition of several 
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experimental techniques aimed at producing 
good performance on the moderately difficult 
recall task (remembering one's chronic condi- 
tions accurately) caused performance to suffer 
in the way the Yerkes-Dodson law predicts. 

If an extensive interview^ such as the one 
described by Laurent, had been used in com- 
bination with reinforcement instead of l^g 
questions, the respondent task would have been 
easier and accuracy would have been increased* 

Effects on specific mofiw^j.— Specific motives, 
such as achievement or social approvals may be 
affected by reinforcement. The tendency to 
achieve or seek approval, in addition to being 
influenced by general drive, can also be height* . 
ened or dampened by the amount of social 
approval given at any particular time. This might 
be made clearer by the following analogy^ If a 
person is hungry and eats a reasonable quantity 
of food, he soon behaves as if he is no longer 
hungry. It is said that the food consumption 
reduced the desire for additional food. If, on the 
other hand, a person is not particularly hungry 
but is allowed to eat a sm^l quantity of food 
(e.g., one potato chip), he often behaves as if he 
has become hungrier. Similarly, social reinforce- 
ment may increase the tendency to seek social 
approval (or avoid social disapproval) and task- 
oriented reinforcement may increase or decrease 
the desire to do well. 

In the Marquis and Cannell study,* ^ some evi- 
dence was obtained suggesting that the respond- 
ent's tendency to seek social approval (or avoid 
social disapproval) was reduced as a result of 
receiving reinforcement. These data are far from 
conclusive^ but if the social approval tendency 
of respondents can be reduced, they have im- 
portant implications for response accuracy* 
According to Edwards,*® people tend to err by 
responding in a socially desirable direction. If 
there is some way to make people less concerned 
about social approval or disapproval in the inter- 
view, interview data would presumably be more 
valid since respondents would be less reluctant 
to report socially disapproved information about 
themselves. 

It may be that positive reinforcement, which 
represents social approval given by an inter- 
viewer, actually reduces the respondent's ten- 
dencies to fear social disapproval and thereby 
reduces his reluctance to report socially dii- 
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approved information. Marquis and CannelP^ 
also found that reinforced reipondents report 
about one and one half times as many highly 
embarrasiing symptoms for themselves as do 
nonreinforeed respondents. This finding is con- 
sistent with the specific motive reduction 
hypothesis discussed above. 

On the other handj it may be that an inter- 
view using only a few reinforcing statements 
may fall victim to the "one-potato^chip effect," 
It may be that fear of interviewer disapproval 
could be accentuated by having only a small 
number of positive feedback statements pro- 
gramed into the conversation. 

Feedback and establishing a relationship, ---It 
is often stated that the success of an interview 
depends on the degree to which the interviewer 
can establish rapport with the respondent. Oper- 
ational definitions of rapport differ and are 
usually unstated. The concept often seems to 
refer to a relationship of personal understanding 
and approval between the interviewer and re- 
spo;ident which is thought to facilitate response 
accuracy. 

The evidence is reasonably clear that feedback 
or other positive statements which do not nec- 
essarily follow any particular contingency sched- 
ule result in the respondent approving of the 
inten/iewer. What is not clear/ however^ is 
whether the resulting positive feeling has any- 
thing to do with obtaining valid data. 

The relationship between approval (produced 
by several types of feedback) and performance is 
Found to be nonexistent, Marshall, Marquis, and 
Dskamp^^ showed that respondents tended to 
like interviewers who made positive comments 
ibout their perform^ce more than they liked 
nterviewers who made only negative comments, 
However, interviewer comment style was un- 
'elated to accuracy of recall even in these ex- 
reme conditions. 

Bales' data^® suggest that, for long-term inter- 
LCtions, feedback statements indicating soUdar- 
tyj agreements acceptance, attention, and satis- 
action, or promoting the release of tension 
erve a "maintenance" function. That is, they 
erve to keep a team or group together and 
working on a task* Presumably, without this 
ind of interaction, task-driented groups would 
reak up without finishing the job at hand. It is 
bt clear, however , that a short-term interview 



interaction requires this soGioemotional kind of 
interaction to be successful* 

Finally, Hymm et al.^^ have questioned the 
assumption that rapport is desirable in the inter- 
view* They point to the possibility that the 
social relationship implied in a high-rapport 
interview may prevent the full disclosure of 
socially unacceptable information, Canneli^* 
has proposed an experimental test of this 
hypothesis. 

Thus, while positive feedback in the interview 
seems to create a good relationship, this relation- 
ship per se may have either no effect or negative 
effects on reporting accuracy* On the other 
hand, it may be that the relationship somehow 
interacts with other variables (such as a condi- 
tioning procedure) to influence response 
accuracy. Further research 

EXPiRIMENTAL STUDY* 

The social reinforcement effect for adult 
humans is fairly well established. In a review of 
the literature, Krasner^^ pointed out that re- 
inforcement effefits have been demonstrated 
under numerous settings with a variety of re- 
inforcing statements, with mmy kinds of 
responses, and with different types of people* 
Currently, attention has turned to a considera- 
tion of other variables which may be ir^ ^ ^nnt 
in producing reinforcement effect:; ' n 
performance. 

One of the mq'or concerns in the preceding 
discussion was the separation of the instruc- 
tional or cognitive effects of reinforcement from 
the more automatic conditioning effects. This 
issue is very important to sumy intemevy plan- 
ners for several reasons and has some major 
implications for theories of human behavior. For 
example, if the reinforcement teclmiques men- 
tioned above achieve their effects merely by 
informing the respondent about what the inter- 
viewer wants him to do, it would seem that 
there are better ways of passing this information 
along to the respondent* Using only reinforcing 
statements to convey information about what 



^This section was written in coUiborttion ndth Ma. linda 
Woodj who hai undertaken the major respoiiiibilitiea of deitgn 
and execution of the experiment desoibed here. 
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the intemcwer wants has the distinct disadvan- 
tage that the respondent can misinterpret the 
message or possibly never get it at all. On the 
other hand, the reinforcement procedures might 
be producing better reporting which could not 
be produced by other means. If this is true^ 
reinforcenient procedures should be considered 
seriously for all personal interviews, as a tech- 
nique for improving reporting. 

This issue of whether the reinforcement effccl 
is purely cognitive or more than that is a major 
concern of experimental psychologists. These re- 
searchers have given a great deal of attention to 
a process they call ''awareness,'' the respond' 
cut's knowledge thut reinforcement is being used 
and that it is used only after he gives certain 
kinds of answers. A number of researchers® ^^^^ 
maintain that the reinforcement effect can be 
obtained lor a human subject only when the 
subject is ''aware" of the response-reinforcement 
contingency or, in terms used here, is aware of 
what the interviewer expects him to do. The 
mere fact that a reinforcing statement follows a 
particular response is not in itself sufficient to 
increase the probability of occurrence of that 
response. An increase can be obtained only 
when the respondent understands the relation- 
ship between his answers and the reinforcing 
statements. The implication of this position is 
that one may obtain high levels of respondent 
performimce right at the start of an interview 
merely by giving clear instructions to the 
respondent rather than by using reinforcement. 
Furthermore, since reinforcement takes a longer 
time to achieve its effects, some respondents 
may never become aware of what is wanted. 

Another group of researchers^ main- 
tain that reinforcement can have a direct condi- 
tioning effect. They do not deny that awareness 
or cognitive effects of reinforcement may con- 
tribute to the reinforcement effect, but say that 
these arc not necessary in order for the re- 
inforcement effect to occur. The major thrust ot 
the research of the Letter group is that reinforce- 
'ment effects can be produced without the 
respondent becoming uw^^e of the eKpcctations 
of the experimenter or interviewer. These 
studies do not show that reinforcement proce- 
dures are more efficient than direct instruction 
procedures. They do indicate, however, that 
human behavior can be changed without the 



necessity of the rcspondciit having to think 
about the change. 

Earlier it has been shown that innovative 
instruction procedures such as brochures, calen- 
dars, and informing the respondent about key 
questions prior to interviewing were not el fee- 
tive in producing better data: Thus, establishing 
awareness through these initial procedures was 
not sufficient to produce desired pcrformLUice, 
as Spielberger and others might imply. 

However, in view of the large number of 
studies supporting each point of view, it may be 
that both interpretations are correct but that 
each is true under different circumstances. It 
would seem that there is some other vanahle 
that helps to determine whether simple knowl- 
edge of what the interviewer expects is sullicicnt 
for good responding ur wheiher more compli- 
cated conditioning procedures are necessary. A 
comparison oC the two sets of studies suggests 
that this vuriable is *Hask difficulty.'' Those 
studies that suggest that awareness is sufficient 
in itself usually involve n simple task, for exam- 
ple, the selection of a first-person pronoun.^^ 
Those that suggest that awareness may be un- 
necessary involve a somewhat more dilficult 
task, such as giving '^self-acceptance" re- 
sponses.^^ Thc^jlifferencc in difficulty between 
these two type's of tasks has been obscured by 
the fact that as long as the interviewer's expecta- 
tions arc unHtaled, both types of task seem dilli- 
cult. 

It must be hypothesixed, iherefore, that the 
conditioning effect of reinforcement brings 
about changes in respondent ability level or skill 
in responding adequately (see ''Dependent Vari- 
ables,'* under '-Design of an Experimental liiter- 
viewing Approach," this report) and that, to the 
extent that a respondent's skills are low, the 
conditioning effects of reinforcement will be 
greater. That is, the importance of awareness, 
however obtained, is relevant to knowledge ol 
expectations, but only reinforcement can change 
skill. When skill needs to be improved, a condi- 
doning procedure is necessary. On the other 
hand, when the task does not require a high level 
— ^of skill, direct instructions should produce maxi- 
mum perfornumce and a reinforcement proce- 
dure sviUnot produce any further improvement. 

Some of the basic questions that arise in the 
consideration of reinforcement eflects should be. 



clarified in order to improve the quality of re- 
porting. For example, 

a. How important is respondent knowledge of 
the interviewer's expectations? 

b. If this knowledge is essential, is it conveyed 
better by interviewer instruction, by re- 
mforcement, or by some combination of 
the two? 



c. Is the recall skill of the respondent im- 
■ proved by reinforcement and docs the 

amount of this improvement depend on the 
difficulty of the task? 

d. Does reinforcement provide information to 
the respondent about the adequacy of his 
responses and does the importance of this 
information vary with the respondent's 
skill? - 



MEMORY AND INFORMATION 
RETRIEVAL IN THE INTERVIEW 



_ The purpose of this section is to present some 
hypotheses about underreporting and to de- 
sqribe an experiment designed to reduce under- 
repartrng ln a field survey. One of the principal 
causes of underreporting is the failure of recall; 
mformation is not reported because it is not 
retrieved from memory. Work in interviewing 
methodology tends to support the assumptions 
u ^„^Geoch23 and modem interference 
theory^* that mformation does not disappear 
from memory but may become difficult to recall 
because of interfering associations. Only the 
accessibility of information declines, resulting in 
a "lessening probability of retrieval from the 
storehouse."" Thus, underreporting is essen- 
tially a problem of retrieval, and reporting may 
be^ improved by manipulating conditions under 
which retrieval occurs. 



THE INADEQUATE 
SEARCH HYPOTHiSIS 

Interviewing methodology indicates that the 
conditions of recall have a crucial effect upon 
, t_He outcome of an interview. Some information 
may be unreported simply because the questions 

poi comey to the respondent an accurate 
notion of what information to search for. For 
example, 19 percent of a sample of respondents 
interviewed in the Hcsdth Interview Survey 
declared in a folIowu|j interview that they 
thought the interviewer was interested only in 
"fairly important" things.5 5 By itself this ob- 
servation provides an alternative hypothesis to 



interpret the underreporting of low-impact 
Items. They may be underreported only because 
the respondent does not consider them to be 
relevant and therefore does not even search for 
them. 

Severd early experimental studies had sug- 
gested that the nonreported material had not 
been repressed or deeply suppressed, nor had it 
vamshed from memory, but often was simply 
not elicited by the usual questioning procedures 
It seemed Ukely that the use of different sets of 
questions and techniques by interwewers could 
significantly decrease underreporting. For In- 
stance, over half the . hospitalizations not re- 
ported m a first interview were reported in a 
second interview.^? An experimental procedure 
which iincluded a few extra questions, more 
explanation of purpose to respondents, and a 
mail followup also resulted in a significant 
mcrease in the reporting of hospitalizationi.is 
Addmg probes to major questioni regarding 
visits to physicians reduced the underreporting 
by 7 percentage points, from 30 to 23 percent.^ 
Another study by Balamuth, et al.* indicated 
that checklists also seemed to reduce the under- 
reporting of chronic conditions. A laboratory 
study showed that the form of the questions had 
a great effect on the accuracy of reporting items 
from amovie.6 7 Finally, an experimental study 
by Marquis, Cannell, and LaurentSa demon- 
strated a substantia] effect of mere question 
length upon the validity of the reporting of 
chronic conditions, 

A major contribution of these studies is the 
demonstration that experimental questioning . 
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strategies can improve reporting by changing the 
conditions under which the respondent is invited 
to search for p^t cventi* While these studies 
surest promising avenues for methodological 
progresSj they are not concerned directly with 
the cognitive processes involved in re^U, 

AN INTlGRATIVi HYPOTHISIS: 
COGNITIVi INADEQUACY OF 
STIMULI QUESTIONS 

Although one can hypothesize that fdlure to 
recall information is an important reason for 
undcnreporting and also demonstrate that rerall 
can be improved^ it is not immediately apparent 
what steps one should take to improve reporting 
in the survey inteiview. Why do customfflry 
questioning procedures fail to obtain adequate 
reporting? Why* for example, if a respondent is 
asked to report his dental visits of the past 6 
months, is he likely not to report them aU? A 
tautologous answer is that for some reason the 
question was not adequate to stim^ate retrieval 
of information located in memory* The in- 



adequacy of a single question to elicit informa- 
tion suggests a consideration of how information 
is copiitively organized in memory. 

When a person experiences an events the event 
is not merely recorded in its original form in the 
mamcr of a computer tape, but rather it 
becomes organized in a perceptual field. As in 
the old iUi^tration of a blind man describing an 
elephant, the meaning of an event depends upon 
how it is perceived and with what other events it 
becomes ^sociated in memory. Thus, what is a 
simple, single-dimcniion variable for the re- 
searcher may not be a simple item for the 
respondent. The dental visit the researcher sees 
m a simple item of information may be 
organized in one of several frames of reference 
by the respondent* The respondent may think of 
the visit in terms of cost, pain, or loss of time 
from work. From this point of view the lingla 
question about the occurrence of the dental visit 
may be ineffective; it may fail as a stimulus to 
re^U the event. 

As shovm in figure 1, the respondent's cogni- 
tive organisation and the researcher's design of 
a questionnaire can be viewed as two diverging 




paths by wliich information is processed inde- 
pendently and quite differently before the inter- 
view. These two paths lead to two independent 
informational states— memory trace and stimulus 
question=whose interaction in the interview is 
expected to produce the retrieval of the initial 
information. This model indicates that the prob- 
ability of proper recall is a function of the 
ability of the stiumli questions to interact 
adequately with the respondent's cognitive 
organization* The appropriateness of the stimuli 
questions is a function of the researcher's ability 
to comprehend the nature of the respondent's 
cognitive path and to utilize this knowledge in 
the framing of the questions. 
, The methodological objective becomes one of 
redesigning questionnaires to facilitate recall* 
One way of doing this is to incorporate into the 
design of questionnaires some of the things 
already known about how people learn, store, 
and retrieve information. The experiment^ de- 
scribed below is an exploratory attempt to 
ajcertain the validity as well as the feasibility of 
this cognitive approach for improving reporting 
in the household interview; 



DESIGN OF AN KXPERIIVIENTAL 
INTERVIEWING APPROACH 

The study was designed to increase the 
reporting rate of acute and chronic illnessesp 
usually underreported in household interviews. 
The strategy consisted of an experimental ques- 
tioning procedure which would provide stimuli 
^.wlevant to the respondcnt-s cognitive processing 
of health information. This procedure was ex- 
pected to improve retrieval from memory and to 
increase the probability of reporting. 

To find out abcrut the sicknesses of a person 
during the month preceding an interview^ one 
may ask a standard qiiesfroh^sueh as^ -^Were you 
sick at any time last month?'* If the person had 
had innuenza and had stored this experience in 
memory as -'being sickj" there is some chance 
that it would be reported in response to this 
question, 

dotatled presentation and discussion of ihU experiment 
appears in a itudy by Laurent, Cannell, and Marquis, 
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If, among other illnesses^ the researcher is 
particularly interested in collecting data about 
influenza, he may also wmt to use a simple 
question such as: "Did you have the flu last 
month?" This question will be a powerful 
stimulus only if the sickness has been experi- 
enced as the flu or influenza and was concep- 
tualized as such. There are clear limitations to a 
straight application of this recognition technique 
in the interview. Aside from the old issue of 
suggestibility^ an exhaustive list of all potentially 
relevant items of information would not usually 
be feasible. Furthermore, the recognition tech- 
nique relies much more than other retrieval 
techniques upon the assumption that researcher 
md respondent share the same concepts. This is 
clearly shown in a medical interview. A physi- 
cian contends that his patients usually report 
only major surgery when asked about previous 
operations, whereas they tend to report both 
major and minor surgery in answer to a question 
about stitches. The recognition principle is basic 
to the recall processes but its application to 
interviewing procedures is valuable only if 
appropriate stimuli of reeognition_can_be_dfc=. 
signed. 

All of this experience indicates that an event 
may be stored in memory under various infor- 
mational states so distant from the initial infor» 
mational state that a stimulus merely traced 
from the original event or from its straight 
conceptualization might not elicit the stored 
information. Last month's influenEa may have 
been stored in memory in many different ways 
and not necessarily as being sick or having the 
flu. There is enough evidence from experiments, 
as well as from everyday experienc€j to show 
that memory is not a simple recording device 
but rather a complex process in which informa- 
tion is transformed and organized. Thus, influ- 
enza may no longer exist in memory as a 
sickness, but may be organized around a number 
of other possible traces. For example, the 
interference of serious chronic illnesses may 
cause the respondent to fail to consider the 
episode of infliienza as a sickness and no longer . 
to store it as such. On the other hand, this minor ' ■ 
flu may have prevented the respondent from : 
going to work on a cold day, thus reducing his 
income in a manner significarit enough to have 



impact on his budget. It becomes clear that this 
respondent may be less likely to report tb? flu in 
answer to a single queition about sickness than 
to a question such mi *'Have you lost any 
income because of any sickness during the past 
month?" If the flu has caused this person to be 
feverish, a symptom that he very seldom experi- 
ences, another relevant stimulus might be: 
"Have you had a fever during the past month?" 

Memory can process an illness in such a way 
that it gets transformed in storage and becomes 
organized around concepts such as pain, ineapaC' 
ity, cost, %^sits to doctors, hospitarizations, 
medication or treatment, symptoms, or more 
generally around other causal, circumstantial, or 
consequential events. The initial perception may 
be distorted to m association with another 
perception in order to fit into some structure. If 
one asks a broad question such as, "Tell me 
about your illnesses^ the respondent has to 
make the effort to review a multitude of 
cognitive structures in order to recall properly. 
He has to invent appropriate frames of reference 
to guide his search; he has to create his own cues 
to reactivate traces of possibly weak salience. 
Altogether, the t^k is enormous and complex 
and the motivation to invest substantial effort in 
it cannot be expected to be spontaneously high, 
especially in the context of an information- 
getting household interview that has no 
immediate benefit for the respondent. Within 
this framework, underreporting is predictable; 
the broad question is not an adequate stimulus 
to the relevant frame of reference and the 
respondent is not going to work hard to recall 
information. 

This framework, however, provides prospects 
for an experimental methodoldgy. Instead of 
asking one standard question essentially traced 
from a simple coneeptualization of the event to 
be recalled, several questions may be asked that 
are traced from hypothesised states of informa- 
tion after memory processing. In other words, 
instead of requesting from the respondent the 
difficult task of building up his own cues and 
frames of reference, the researcher should at» 
tempt to create these recall aids and to build 
them into the questionnaire. If the researcher 
can be successful in predicting and designing the 
relevant cues and frames of referenGC, then the 
r€Spondent*s recall process should be signifi- 



cantly facilitated and the availability of informa- 
tion correspondingly increased. 

The Extensive Quiitionnaire 

The strategy of an extensive health interview 
consisted of designing a questionnaire containing 
a large number of questions that would provide 
the respondent with multiple and overlapping 
frames of reference and cues (see Laurent, 
Cannell, and Marquis^®). Medical information 
was asked within classical conceptual frame- 
works as well as in the language of the layman, 
and through standard questioning as well as 
through multiple behavioral cues or direct recog- 
nition of items. Transitions between sections 
were also used to bring some relief in the 
questioning style and to instill a deliberately 
relaxed pace in the interviewing. 

The questionnaire started with question about 
symptoms, such as, "Do you have pains in the 
abdomen?'* "Have you had any pain or soreness 
in your joints?** "Have you had trouble breath- 
ing?" Every time the respondent gave a "Yes" 
answer the interviewer used the probe, "Do you 
have any idea what causes it?'" in an attempt to 
obtain the report of an underlying health condi- 
tion. Other frames of reference and cues were 
used, such as asking for a medical history by 
rtieans of queries related to childhood, adult- 
hood, 6 or 12 months previous, last week, or the 
week before last. Then specific behavioral impli- 
cations of illnesses such as diet, food sensitivity, 
restrictions of activity, medications taken, and 
visits to physicians were all used to provide 
assistance in the retrieval process. Finally, a 
checklist of chronic conditions implemented a 
direct items=recognition approach. 

Control Questionniirt 

A control questionnaire was used in the 
collection of information on the same major 
items of health information as the extensive 
form, but it consisted of single direct questions 
for each variable. This procedure contained 
standard questions comparable to those used in 
the Health Interview Survey and the same 
chronic conditions checklist that was used in the 
extensive form. 



Field Experiment 

The study was designed to compare the 
effectiveness of the two questionnaires on two 
mdn dependent variables: (a) the number of 
health conditions reported, and (b) the impact 
of these conditions on the respondent. 

To decrease the varimce from factoire other 
than those puiposely introduced by the experi- 
mental design^ the sample population was 
homogeneous-a restricted segment of English- 
speaking, native-bom, white females between 18 
and 65 years of age, all of whom were residents 
of the city of Detroit and were of low-middle 
and middle socioeconomic status. Two clusters 
of three dwelling units were chosen at random 
from each of 110 blocks, selected with probabil- 
ity proportionate to size in 16 census tracts. 
Only one person in each dwelUng was . inters., 
viewed; the wife in the household the first 
choice. Six female interviewers w*ere employed 
in the study and were assigned to geographically 
convenient sections of the city. The assignment 
of experimental questionnairei to households 
wa^ random within each sample block. 

The study showed that 204 respondents were 
interviewed, consisting of 105 extensive and 99 
control subjects. Aside from the variations in 
questions^ the interviewing techniques were kept 
constant for all subjects (e,g,j introduction of 
the survey, getting demographic information, 
probing procedures, interviewing style). All 
respondent demographic characteristics were 
similar, as were the response rates (87 percent in 
both interviews). On the average, the duration of 
the extensive interview was 74 minuter and of 
the control interview, 40 minutes* 

Dependant Variables 

Eligible health condition.— To ensure the com- 
parability of the data collected through the two 
interviewing procedures, precise criteria were 
established to determine the eligibility of any 
reported health problem as a condition. Every 
time any health problem w^ mentioned in 
either of the two procedures, standard struc- 
tured probing was used to ascertain its eligibility 
as a condition. In order to be m eligible 
condition for data analysis, an illness had to 
present either acute chamcteristici— that is, to 
have started within the 14 days preceding the 



interview— or chronic characteristics— to have 
chronic implications for the person*s health* All 
conditions were screened by the intemewer and 
later by the coder to include only those meeting 
the criteria of the Health Interview Survey, The 
number of eligible conditions reported was the 
m^'or dependent variable analyzed* 

Condition impacL—AB suggested earlier, 
health events get transformed and organiEed in 
memory within various clusters or frames of 
reference* It is assumed that the number of such 
categories under which a health condition is 
organized is a mark of its salience or impact and 
a predictor of its accessibility for retrieval* A 
condition of low impact that is organized under 
only a few categories is likely to be missed by a 
single question. Therefore, the hypothesis in this 
study was that the extensive interview would 
pick up more^onditions of low impa^ 

To create a measure of impact in both 
techniques, every eligible condition reported was 
probed according to a standard procedure. Addi- 
tional information was asked about the exist- 
ence of health behaviors associated with each 
condition occurring during the past 2 weeks 
(visits to a physician, medicine, treatment, spe- 
cial diet, days in bed, days of restricted activity, 
amount of pain or discomfort). An index of 
impact was created for each reported condition 
on the basis of this additional information. 

The two dependent variables were selected 
upon the empirical evidence that underreporting 
represents a mq'or problem in the health inter- 
view. As medical records were not available for 
this research, a working assumption built into 
the design of the study was that the more 
information reported, the better the overall 
validity. 



Hypoth^ii 

The major hypotheiis that the study at- 
tempted to test was the following! By providing 
a broad aid to the respondent's recall process 
through the use of multiple frames of reference 
and cues, the extensive procedure is expected to 
increase the overall number of reported elipble 
conditions as compared with the number ob- 
tained in the control procedure. Since events of 
low impact are more likely to be underreported 
than those of high impact, a significant increase 
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was particularly expected in the reporting of 
chronic conditions of low impact within the 
extensive interview* As a coniequence, the over- 
all impact level of the reported conditions was 
expected to be lower in the extensive than in the 
control procedure. 

Results and Disouislon 

As shown in table 38, the data clearly 
supported the major hypothesis. A significantly 
larger number of health conditions was elicited 
by the extensive interview than by the control 
interview. Respondents reported an average of 
7*88 eligible conditions in the former and 4.42 
in the latter. The increase in reporting was 
significant for both chronic and acute condi= 
tions. 

As. predicted, the increase in reporting was 
obtained by eliciting conditions of low, but not 
trivial, impact that were not reported in the 
control interview. As shown in table 39s the 
mean level of impact for all conditions was 



sipiificantly lower in the extensive procedure 
(2*03) than in the control procedure (2*64), and 
this was especially true for chronic conditions. It 
was further verified that the reporting of higher 
impact conditions wm similar for both tech- 
niques* 

Several analyses have been conducted in an 
attempt to identify the main sources of im- 
proved reporting in the extensive questionnaire. 
It w^ found, for instance, that 61 percent of all 
conditions reported in the extensive interview 
were elicited by the initial symptoms questions. 
These, questions dealt with the presence of 
specific symptoms, aches, or pains and with the 
health conditions underlying them. The average 
number of conditions reported under these 
ihitial questions in the extensive interview was 
larger than the total average number of condi- 
tions reported in the entire control interview*^ 
This observation indicates that a cue-giving 
approach using symptomatic mmiifestations of 
illnesses as a frame of reference is more produc- 



Table 38, Mien numbtr of hialth conditlQni riporttd per ptrsgn and diffirfnoe batwisn means^ by type of condition end 

queitionnalre procedure 



Reporting varrabla 


Mean number of 
oondltlons rfported 


Difference 
between 
mians 




iKteniivi 
proceduri 
(105 respondents) 


Control 
proceduri 
(89 reipondinti) 


All aliglble health conditions . > 


7.88 


4.42 


3.48 


.001 


Chronie conditions ^ ^ ....... ^ / ... . . 

Acuts conditions (in last 14 days) .................. 




3.99 
0.33 


2.30 
0.49 


001 
.001 



^Slgnif leaned tevel of difference. Values were computed on the basis of the one-tailed f-statritlcs^ 



Table 39. Number and mean Impact level of health condltloni reported, by type Of condition and questionnaire procedurt 



Reporting variable 


Number of health 
condltloni 


Mean Impact level 


Oiffertnct 
between 
means 




Extensive 
procedure 


Control 
procedure 


Extensive 
procedurt 


Control 
procedure 




All eligible health conditions .... ^ ....... . 


661 


399 


2.03 


2.64 


—0.61 


.001 


Chronic conditions ^ \ i . . i ^ i . . . ^ ..... . . . 

Acute conditions iln last 1^ days) ............ 


Bi3 
73 


HQ 
' 29 


3.34 


2.46 

4 93 


^0.59 
-1.69 


.W1 
.025 



Significancs tivel of differenof. Valuei were computed 



on the basis of the one'talled f'Statlstlci. 



tive in eliciting the report of illnesses than are 
the standard questions actually designed to ierve 

- this purpose- 

Another observation of this kind concerns the 
effectiveness of recognition checklists. At the 
end of both interview procedures^ a standard 
^ checklist of 41 chronic conditions was utilized 
and the relative effectiveness of this device was 
evaluated within each procedure. It was found 
that this last section was highly productive in 
the control questionnaire (57 percent of all 
conditions were first reported there) and appre- 
ciably less productive in the extensive question- 
naire (where it yielded a total of 16 percent of 
/first reports). The high figure obtained under the 
control procedure (57 percent) confirms to 
some extent the effectiveness of an items recog- 
nition approach as , a basic tool for eliciting 
information in a standard type of interview. The 
lower but still substantial figure obtained under 
:the extensive procedure (16 percent) demon- 
strates the adequacy of this recognition ap- 
proach for gathering information not already 
reported through the various cues provided 

- during the course of an extensive interview. 

vi A comparison of the differential effect of this 
recognition approach under the extensive and 
control interviews for the reporting of chronic 
conditions is shown in table 40. In the extensive 
interview^ 67_percent of the chronic conditions 
in the final standard checklist were first elicited 
before thif final checklist^ as co^mpared to only 
26 percent in the control intemew. In both 
techniques, the high-impact conditions were 
r reported first, the low-impact ones last. This is 
especially typical in the control technique where 
the mean impact level varies from 4.36 for 



chronic conditions reported prior to the recogni- 
tion list to 1*61 for those reported on the 
recognition list. An analysis of variance carried 
out on the impact level of all eligible conditions 
reported by chronological sections of the ques- 
tionnaires in the two procedures led to the same 
observation, showing a significant lowering of 
impact from the earlier sections of the question- 
naires to the later sections. 

The same finding was replicated on a question 
basis in the extensive interview. The extensive 
interview made some use of primary or standard 
questions followed by additional or cue-giving 
questions* The average impact level of eligible 
conditions reported for each of these two types 
of questions is shown in table 41. An analysis of 
variance based on these data showed that the 
average impact score varies significantly 
(p<.05) according to whether the condition 
was reported in the primary' or additional 
questions* Table 41 demonstrates the tendency 
for the averse impact to be lower for condi- 
tions reported on the additional questions (2.25) 
than on the primary questions (3,66). This table 
also illustrates clearly the power of additional 
cue-pving questions to elicit information not 
reported on standard questions. In this case the 
additional questions are providing as mmy and 
even more eligible Conditions (36) than the 
prim^ ones (32). . 

This general tendency for high-impact condi- 
tions to be reported earlier and for low-impact ^ 
conditions to be reported later— within an entire 
q uestionnaire or within given questions^ 
confirms the idea that high-impact events are ^ 
more easily recalled and reported than are 
low-impact events. Since they are -easier to 



Table 40.. Perctnt of chrDnIo conditions riported and mtsn impfct level ©f lifted chronie eonditloni, by uvhether first riported prior tb 

or in responsa to recognition fist« by questionnairi procedure 
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Qu^tionnalre procedure 


1 Listed chronic conditions 


' Total listed chronic 
conditions reporttd 


Pint reported prior 
to recognition list 


First reported in 
^reiponsato 
recognition lift 


Percent 


Mean level 
of impact 


Percent 


Mean level 
of impact 


Pereent 


Mean level 
of impact 


intensive * . 

Control . . . , . » . . , . . , . , , ^ . ..... . . . . 


67 
• 26 


4.36 


33 
74 


1.4S 
1.61 


100 
100 


2.02 
2,32 



69 
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Table 41. Number and impact level of elfgible conditions, by 
. wbe^tr first reported in primary or additional qutstions of 
.the extensive questionr^ire 



Questions In which 
eondition first 
- reported 


Number 
of 

conditions 


Impact level : 


Mean 


Standard 
deviation _ : 


Both typeiof 
queitlons * . . . * . * 

Primary questions * . * * \ 
Additional questions rr^' 


88 


2.91 


2.36 


32 
36 


3.i€ 
2.25 


2.SS 
1.S7 



NOTE: F^B,BQ{p< m), 



recall, they are reported first; low-impact items 
appear harder to obtain from the respondent 
and, as such^ require stronger stimuli (recogni- 
tion lists or cues) and more time. Thus the 
recent behavioral implications, or impacts of an 
event strongly affect the likelihood of its re- 
trieval* This likelihood seems to increase as the 
amotint of behavioral implications increases, and 
vice versa. ^ 

The inadequacy of standard interviewing to 
elicit reports of lower impact events appears, 
then, as a major cause of underreporting and 
incomplete data. The analysis presented above 
has pointed to the ability, of some intersdewing 
devices to improve the prospects of reporting for 
events whose behavioral implications arc weak. 
It is interesting to note that in spite, of its 
low-impact character j the additional information 
collected through the extensive technique w^ 
still significant in terms of the criteria of the 
Health Interview Surveys a§ it involved, for 
example, restricted activity or use of medical 
services, . .. . 

Conclusions 

The results of: this study- emphasize the 
important effect of question content and ques- 
tion strategy on survey data and suggest several^ 
methods of reducing underreporting bias. The 
nietho do logicar objective was^ to demonstrate 
that the completeness of the reported informa- ^ 
tion can be changed significantly by programing 
the respondent"S recall task more efficiently* 
Standard questions may not represent the most ■ 



adequate stimuli to activate respondent recall 
because they may ignore the way in which 
information is organized in memory. Thus an 
attempt was made to pattern an experimental 
questioning procedure after the processes that 
the respondent was expected to use in acquiring, 
storing, and retrieving information; This was 
accomplished by the use of multiple frames of 
reference and mtdtiple cues integrated into a 
questionnaire* A substantial increase in informa- 
tion was obtained through these procedures in 
the areas of information where underreporting is 
traditionally observed. This improvement is 
interpreted as the result: of a greater corre- 
spondence between the questioning procedures 
and the mmner in which respondents organize 
health inforination in memory. 

Several uncertainties remain, however* While 
thf re i s evide nce from the reported data that the 
additional information elicited in the extensive 
procedure is not trivial, the validity of - this 
information needs to be ascertained by a . study 
which would check" the respondent's reporting 
against valid records. The amount of overreport- 
ing obtained with the. experimental technique 
should be evaluated ^d compared^ with the 
amount obtained using a control technique. 

On a more theoretical level, : although the 
results of this study supported the hypotheses, it 
is not possible to infer from these data any 
satisfactory statement of causality. Indeed, the 
interactions between cognitive and motivational 
factors involved in the intemew situation were 
not controlled in this study* Even though the 
approach was cognitive,: it seems that motiva- 
tional changes also occurred that may have been 
instrumental in determining the outcome of the 
experimental treatment. For instance, as the 
extensive procedure made the recall task easier, 
it also reduced the amount of effort required to 
perform it and thus reduced the motivational ^ 
requirements of the task as well. Also, since the 
interviewer had to devote more time and display 
more behavior in administering the extensive 
questionnaire than irf^^dministering the control 
procedure^ „ this increas^ activity may have - 
conveyed^the idea that the^task^ was important; ' 
and may have thus heightened the respondent's ' 
motivation to perform* Too, the respondent 
may have modeled his behavior after that of the 
interviewer (the intemewer did more talking in 



the extensive interview and the respondent may 
.have followed his lead by being more active and 
reporting more info^ It is- unclear 

;> whether increaies in reporting have been ob- 
:^ ; tained through direct cognitive facilitation^ re* 
jduction of motivational requirements^ indirect 
: motivational stimulation J or a combination of 
these factors. The major QUtcome was a prag- 
matic one; techniques designed in a cognitive 
: framework to facilitate recall have proved efhc- 
; tive in inCTcasing reported infdrmation, 
: Finally, two main implications deserve partic- 
•i ular attention. First, it s the complex- 

: ity involved in retrieving adequate information 
in an interview tends to be underestimated. Even 
data that appear relatively simple to obtain may 
require the design of more elaborate interview- 
ing techniques; Efforts have been made in past 
research to overcome various biases in interview 
responses. Howeverj in most cases they have 
; been avoidance strategies rather than construc- 
tive approaches. Evidence from data such as 
those presented above suggests that probably 
, very little is known} about the asking of appro- 
; priate questions, so that reporting errors may 
often be the result of questioning errors. In view 
of the wide utilization of survey methods to 
; create new knowledge and to guide poHcieSj 
: there is great need for more basic research on : 
; the interviewing tool itself. A major objective 
for these research efforts is simply improvement 
in the design of questions. The present rcrsearch 
^ has attempted to demonstrate that: progress can 
be/made in this direction by framing questions 
^ in accordance with the respondent's cognitive 
processing of the initial information. 

A second implication of thii that 
^ the experimental survey research interview could 
provide a new approach for investigations in the 
field of cognitive psychology. If hypotheics 
about the cognitive pro ceis can be introduced 
into the design of interviewing experinients. 



then the interview setting can serve as a labor^» 
tory for the study of humai memory and recalL 
^ It is interesting to note that most of the 
knowledge related to the memory process has 
been developed in classical experiments by 
manipulating the input or learning conditions 
and evaluating the resulting output or recalL 
From this experimental desigiij inferences are 
made about cognitive processing and memory. 
ThuSs in the laboratory the focus is most often 
on input or learning conditions. Little attention 
is given to recall, wUch is usually coriiidered as 
an jend result variable or as a test of learnihg and 
retention after memory processing. Textbdoks 
are m ore likely to dis cuss the psychology of 
learning than the psychology of recalL A re- 
versed strategy was attempted in the present 
researchr the learning conditions or inputs w^ 
kept constant or controlled by experimental 
survey design and conditions of recall were 
manipulated. Recall was no longer considered 
only a test of learning but was viewed <a a 
powerful intervening process itsel^one: which 
mediates the affects of learning a.nd membry 
processing on survey interview reporting, 

*"The. more questions one asks about a topic^ ^ 
the more information one obtains," is a state- 
ment made frequently by survey resekrchers, 
"Recognition4ist; questions will obtain more 
^ information thm those requiring free recall," is; : 
commonly stated by survey practitioriers. The 
present study helps to provide some understand-; 
ing of the phenomena underiying these stated , 
ments. It is not a simple matter of ^king more " 
questions; nor . ^ are v reeogni tiqn-Ust questions S 
necessarily better than other kinds of questions. 
So me basic principles of memory r and = retrieval 
can be used to improve reporting. This study 
suggests that further research will yield a better 
understanding of the :way in which information 
is stored and will invent more effective methods 
of retrieving that information. 



QUESTION LENGTH AND REPORTING^BEHAVIOR 
yj:HEJNTERVlEW:lRREUMINARy IfSIVE 



: This section reports on two exploratory 
studies of the effects o f question length on the 
amount and validity of information reported in 
Iwusehold interviews. Both o f these studies areZ 



described in more detail in Vital and Health 
S^^i^iw; Series 2-Number 45,*3^^ find-, 
ings ^ of the: second one have been discussed 
previously in this report in relation to the effects 
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of reinforcement on household interviews in the 
section, "The tf??^ of Verbal Reinforcement in 
Interviews and Its Data Accuracy." j 

Statements such as the following can be 
found in questionnaire methodology mMuals: 
"Make the questions as concise as possible v , . . 
The length alone m^esjt practically impossible 
to carry the question as a whole in mind*" -^P "It - 
is generally best to keep : questions short- 
preferably not more than 20 words/ * On the 
basis of these statements it was assumed: that 
lengthy questions were an obstacle to f clear 
communication. However, some empirical find- 
ings showing the presence of a verbal behavioral 
balance in the household interview® ^ along with 
other - findings repeatedly demonstrated a matchr ^ 
ing effect between interviewer and respondent 
speech duration.^ ^ ^ From this ^-finding it was- , . 
concluded that long questions migh t have some ; 
value not previously anticipated, 

tmpiricai Findings on Bahavior Matching In 
tha Interview 

The Survey Research Center study by 
CannelU Fowler, and Marquis^- showed a cleaTj ;/ 
positive-association between the overall amount 
of behavioral activity of the respondent and the 
number of items reported. Furthermorei a very 
high correlation was found between the behavior 
activity level of the interviewer and that of the 
respondent* These findings led to speculation 
that interviewer and respondent sought . and 
perceived -cues from each other about the degree 
of effort to put into their respective roles. If this 
cue-search process causes the respondents to 
model rhis ' behavior after: the behavior of the 
interviewer, inferences can be made about ques- 
tion length. Logically, short questions should 
elicit short answers and long questions should 
yield long^ responses. 

This inference is supported by a series of 
studies on interview speech behavior conducted 
by Matarazzo et al.^^ Briefly stated, these 
studies demonstrated that some formal measures 
of the interview i process unrelated to content— 
namely, interviewer and respondent speech dura- 

t:r V-^This study Is desafibed in ^cater detmU in an earlier section 
of this report, "Behavior in Interviews^" For a fu^ report of the 
study, sec reference 46- - - - * ' 



tion and silences— are remarkably reliable^ valid, - 
and consistent. These studies have shpwn very 
explicitly and repeatedly in employment inter- 
views that an increase in interviewer average . . 
speech duration resulted in a significant increase 
in respondent average" speech duration* . For v 
instance, in . a 45-minute interview divided into 
three" 15-minute periods where the interviewers 
spoke in utterances averaging 5*0, 1 5 ,2,; and 5,5 
seconds, the respondents* - utterances ■ averaged % 
30,9p 64.5,' and 31.9 seconds, respectively. In 
other experiments of this series, the researchers 
varied the schedule of the interviev^er sequence 0 
of utterances both in range and direction. They 
also controlled for number and type of ques-^ 
tionsj for topics discussed, and for interviewer 
differences. In all cases: they consistently .ob=^j( 
tained changes of approximately lOQrpercent ini^ui: 
respondent speech duration. The changes were J 
dways in the direction of the patterns shown by : ■ 
the interviewers/ ^ : 

The results, of these studies ..of behavior 
matching in the interview demonstrate; that 
increases in intendewer verbal activity produce 
increases in respondent verbalization. The proba- - 
bility that a long response, will contain more_ 
information is an interesting hypothesis! to be 
tested. The research described here is devoted to— ^ 
an investigation of this problem, 

Hypothisas About the Effects of Question 
Length on Re^rting Behivior 

A major contributidn of the work by 
MatarazEo and his associates was the focus on^ S? 
the measures of speech behavior in the interview 
not related to . content, particularly interviewer 
and respondent speech ; duration. Survey re- 
searchers are interested in the potential effect of 
these vari^les (question length or speech dura- , 
tion) upon the content variables (amount and^s 
validity of reported- information)* Ways: of in- - ^ 
ducing gre ater rasp ondent verbahza tion are ^ o f ^ • 
particular interest because it might be conducive ^ 
to improving reporting accuracy. 



''Replications of this finding hmve been ob tamed under other . 
int^viewing situatioEis, such ai the astronaut-pound cominimi».^^j 
cator convemtionsl^ the Kennedy news conferences and rg 
the experiment on interviewer style by Heller, Davis7i^and--T:^ 
Myss/S 



72 



61- 



One needs to ascertain if the speech-matching 
effect is present Jn_ survey interviews. Fu'^Vaer- 
I more, it is necessary to find out whether the 
: : increases in respondent speech duration reported 
in other studies are a direct result of increases in 
v j interviewer ■ speech duration or the result of 
greater information demands made upon the 
respondent by more elaborate questions. If 
person A is asked the short question "Tell me a 
little bit about your job," and person B is asked 
the long question 'Tell me everything you can 
: think of about your job. I am interested in as 
many details as you can provide to describe it/' 
; one expects person B to talk longer than person 
V A simply because of differences in the demaiid 
characteristics of the question. In order to find 
■ out whether changes in respondent speech dura- 
„ tion are a junction of changes in interviewer 
speech duration and independent of changes in 
; the information demands i of the question, one 
needs to modify the length of the question while 
holding constant the amount of information It 
c requests. Such a design was used in the first 
7 experiment to be described in this section. 

A second -crucial task is to find out whether 
- increases in question lengthy withou 
the information demanded, have any effect on 
the answer contents independent of variations in 
respondent speech duration. A mere increase in 
question . length might result in an increase in 
amount of reported information, with or with- 
vout ;: a corresponding increase in respondent 
speech duration, This hypothesis is based upon 
the Gue*search; model of the interview described 
above, in which it is assumed that the respond- 
ent looks to the interviewer aa a source of cues* , 
A long question might provide the respondent 
.with cognitive and motivational cues conveying 
: the idea that a full report is desired. Italso gives , 
the. respondent more time to work .on recall 
while the long question is being asked. The final 
reporting performance might change, although 
this change might not necessarily be reflected in 
the length of the answer. 

Finally, it is important to ascertain the effect 
of - question length upon the validity of the 
reported information, Validity'^n^ight" be^ im-^" 
proved by the use. of long questions, since the 
interviewer cues might transmit a request for 
completeness and accuracy of report. For this 
step of the research, the respondent's report will 
be checked against independent records. ' H 



EXPERIMENT 1: EFFICTS OF 
QUESTION LENGTH ON ANSWER 
^ 1 DURATION AND REPORTING 
FREQUiNCY 

A pilot field experiment was designed to test 
the effects of interviewer speech duration upon 
respondent speech duration and reporting fre- 
quency*^^ In order to increase control over all 
aspects of interviewer verbal behavior^ variation 
in speech duration was created through the use 
of questionnaires with short and long questions* 
Furthermore, the lengthening of short-form 
questions w^ implemented in such a way that 
the information requested in short and; long 
questions was kept constant. TWs strrtegy . was 
used to rule out the possibility of obtaining 
longer answers to longer questions only because 
these questions explicitly asked for more infor- 
mation. - 

Quiitionnaire Procedures 

An interview containing 28 questions was 
created according to customary methodological 
principles. An average of 14 words was used in^ 
each question. Information was requested on 
various health events (e,g,j acute ilhicsses, in-' 
juries or accidentSs and chronic conditions) and 
health-related behaviors (e.g., medicines taken ^ 
and doctor visits) which occurred during various 
periods ■ of time (e,g*, last 2 weeks, 4 weeks, 6 
months)* The questions were primarily about 
the respondent herself (24 questions) and sec- 
onilarily about another selected member of the 
household (4 questions). The types and numbers 
of questions used were as followsr 

a* Open-ended type, leading to free response 

■ . 

b, "How mmy * type, seeking information 
on frequency of specific events (8); and 

c* Closed, forced-choice, checklist type, deal- 
ing with presence or absence of specific 
chronic conditions (12). 

Then, each question was written in a long 
form according to the following procedure. A 
long question consisted of three sentencesr T''^- 

a* An introductory statement giving a partial 
description of the topic; of the^^question, 
: including the same terms ^as used in the 
short question, but in a different grammati- 
\ cd structure,' possibly using a cliche; ' 



^ b. An intermediary statement conveying in- 
formation already contained in the short 
. . question but not presented in the introduc- 
tory statement, and usually introduced by 
another cliche; or a filler, introducing some 
extraneous informa.tion of obvious and 

■J. inconsequential nature about the survey 
: but unlikely to affect the: meaning of the 
question; and, 

. c. The question itself in its short form. 

The two follovy^ing examples provide an 
illustration of this question-writing procedure: 

: Short Form • Would you tell me what 

accidents or injuries you may have had during 
the last six months? (17 words) 
: 4. Long Form: VJe would like to go next 
to a question about accidents and injuries* In 
this survey we ask everybody to report their 
accidents and injuries for the last six months. 
Would you tell me what accidents or injuries 
you may have had during the last six months? 
(47 words) - 

. 17. Short Form: Hbvg you ever had any 
trouble hearing? (7 words) 

Q. 17. Lori^ Form; Trouble hearing is the last 
item of this list* We are looking for some 
information about it. Have you ever had any 
trouble hearing? (24 words) 

Thus, length was added to questions by 
introducing redundancy, cliches, and extraneous 
information. It was assumed that this procedure 
did not alter the objective or meaning of the 
question* The short form jof the question always 
appeared with identical wording in the last part 
of the long question- Long questions contained ^ 
an average of 38 words each. They were 2.7 
times longer than the average short question* 
This length ratio was assumed to be large enough 
to ensure a substantial variation in interviewer 
speech duration, despite expected variations in 
the speed of reading.^ , 

Three questionnaire procedures (A^ BjandC) 
,were designed from a pool of 28 questions, all 
written in both; short and long forms. The 28 



'^Ray and Webb's research on Kennedy news conferences^^ 
used the number of lines of transcripted speech as the unit of 
speech, duration analysis^: That . measure has been found by 
Matarazzo^^ and othere to be hi^ily correlated with standard 
methods of time recording (over;.90) and highly reliable, " ^" 



questions were used in the same order in all 3 
procedures. Questionnaire C (control) consisted 
of short-form questions only. Questionnaires A 
and B consisted of blocks of long-form arid 
blocks of short-form questions, alternated in 
such a way that each block of questions asked in 
the long form in procedure A was asked in the 
short form in procedure B and vice versa* This 
particular arrangement was used for two reasons- 
Firstj it was assumed that a questionnaire 
employing only lengthy questions might be 
detrimental to useful respondent performance*^ 
Second, this particular design allowed for an 
investigation of potential carryover effect of 
length from blocks of long questions to blocks 
of short questions*** Finallyj this design allowed 
for a comparison of answers to all questions in 
short form in treatment C and long form in 
either treatment A or B (with a mixture totaling 
about one-tliird short questions and two-thirds 
long questions). Thus, the three experimental 
procedures presented the following question- 
length composition r 
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Field Procedural 

Two female interviewers on the Survey Re- 
search Center staff each received about 8 hours 



^Current Survey Reiearch Center w^rk tends to diicard this 
assumption, at least for interviews of moderate total lenfth ;' 
(about IB minutes)^ - _ . . 

"The carryover effect was not found by Mataraizo, but wis 
su^cstcd by the Ray and Webb study of Kennedy news 
eonfcfences. 



of training. They were instructed to read the 
; questions exactly as worded on the question* 
: naireSj to avoid any obtruiive speech behavior, 
and to provide clarification by repeating only 
v^;; the relevant part of the question and only when 
absolutely neceisary* They were also told to 
omit probing, to accept the respondent's answer, 
and to eliminate any kind of verbal or nonverbal 
feedback. they were trained to 

adopt a regular speech rhythm, consiitent for 
v questions of varying length. They were told that 
the purpose of the study was to experiment with 
various types of question structure. 

Four city blocks were selected at random 
^ from two census tracts in Jackson, Michigan, 
The tracts contained white families of moderate 
mcome, with a high proportion of native-bom 
citizens and a low proportion of persons over 65 
years of age* Two blocks were assigned to each 
interaewer and interviews were taken according 
to a random selection of dwelling unit numbers. 
In order to be eligible, respondents had to be 
white, 18 to 64 years of age, married, femde, 
able to respond adequately, and fluent in 
English. \ V 

A total of 27 interviews were taken (9A, 9B, 
and 9C), Questionnaire forms were randomized 
among the four blocks, both in terms of number 
and administration order* Intemewer 1 took 
4A, 5B, and 5C forms; interviewer 2 took 5Aj 
4B, and 4C fornis/ All interviews were tape 
recorded with cassette-type machines, 

Dipendant Variable 

: Two dependent variables, assumed to be 
affected by^ the length of the question asked, 
were measured: 

a. The duration of respondent answers to 
each question* This measure was defined as 
-the number of seconds from the end of the 
a: V : question to the end of the response minus 
any irrelevant interruption or^ any addi- 
; tional interviewer verbal intervention.^ 
b* The percentage, of questions in which one 
;: : or more items of the requested health 
: in fonnation were reported* . 



R^ltt and Discussion 

Answer dura twn, —The ^^^ B^^ number of 
seconds the respondent took to. answer a ques-r 
tion in relation to question length is shown in 
table 42^T wo converging findings emerge. First, 
it is clear that within interviews containing both 
short and long questions (A and B) the long 
questions did not elicit any longer answers than 
the short juestioris did (5.6 seconds against 5.7 
seconds). Second, interviews with only short 
questions (C) .did not elicit substantially shorter 
answers than long questions did in the other 
interviews (A and B). The average answer length 
is slightly lower for short questions used alone 
(5.3 seconds) than for long ones used in coihbi- 
nation (6.6 seconds), but the difference is riot 
statistically significant and may be considered 
inconsequential* : 



Table 42. Number of refipondtnts, number of queitions asked^ 
ifld average duration of insweri per question, by queitidn 
" ■ length "- " : " ■ ' \ 
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Differences between these flgyres are not itatiitically slgnlfN 



cant (two-nlled r). 



^Answer duration was timed from the tapes by a sin^e coder 
using electrohlc timers. 



These results are far from the, approximately 
lOO-percent increase in answer duration re- 
peatedly obtained in other research where com- 
parable lengthening of interviewer ipeech had 
been used. In the limited exploration iinder* 
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: taken, the matching effect between interviewer 
and respondent speech duration did not appear. 

r iti could be argued that in contrast to other 
types of interviews which usu^ly include many 

?^pen"*questionSj a survey interview containing a 
substwti^ number of closed questions does not 
provide a chance for respondents to do much 
talking; therefore, the matching effect docs not 
emerge. 

To provide some understanding of this issue, a 
further analysis of answer duration was con- 
ducted which controlled for question type (open 
or closed)^ While -the average answer duration 
was longer for open questions than for closed 
questions, the matcUng effect of question 
length was not more apparent for the former 
thm for the latter* These results do not replicate 
the MatarazEO findings described earlier. 

In contrast with other researchi the major 
change introduced in this quest ion^type experi^ 
ment was a strict control on the information 
load and demands of the questions, so that a 
long question would not transmit or ask for^ 
more information than would a short question* 
In view of the results and pending replication, it 
is very possible that this control was responsible 
for the discrepancy between the present data 
md the Matarazzo Hndings. The absence of a 
length-matching effect under the new experi- 
mental conditions supports the authorsV inter^ 
pretation of the findings by MataraEzo and 
others, proposed earlier as hypothesis* Varia- 
tions in respondent speech duration may be 
quite independent of variations in : intemewer 
speech duration, resulting instead from changes 
in information content and demand of a ques- 
tion. When information level is held constant 
across short and long questions, there is no 
longer any clear indication of a matching effect 
of speech duration. 

Reporting frequency, --The effect of question 
length upon the probability that responses con- 
tmn one or more items of requested health 
information_ is shown in table 43, It is apparent 
from the table that within intemews containing 
both short and long questions, the length of a 
question does not seem to predict the frequency 
of report; Thirty-eight per cent of the short- 
form questions elicited information, compared 
to 40 percent for the same questions written in a 
long form* The difference is inconscquentiaL On 



Tible 43* Number of re^ondenti, number of queitions asked, - 
and per^nt ef questions for whieh any naqutsted health 
information wm reported^ fay quest len^h „ ~ 
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- Both of these proportions are sign if leant iy different from 
the proportion obtained In the third group (p ^ .05, one-tailed^ 
bmdonZ}. 



the other hand j in interviev/s using only short 
questions^ only 29 percent of the /questions 
elicited lome health report. This proportion 
differs significantly (p < *05) from the propor- 
tioni obtained when both short and long ques^ 
tions were used in the same interview* ; 

Thui, the lengthening of half the questions in 
questionnaires A and B is responsible for a 
significant increase in the^ frequency I'of report, 
compared with the frequency obtained in the 
control questionndre C. Further analysis of the 
data by question type showed tWs effect to be 
present in answers to both closed and open 
questions* 

Two suggestions emerge from the data shown 
in table 43. Firsts the probability of report may 
be, enhanced by the use of longer questions.: 
Second, this effect may be carried over from i 
long to short questions when blocks of both 
types are used in the same intendewrThusjrwhile 
there seems to be a positive effect of question 
length per se upon the probability of report, this 
effect also appears throughout the entire inter- 
view. 



: In summary, the results of this pilot experi' 
: m offer the following proposition. When 
information demand is held constant, long ques- 
tions do not produce noticeably long responses, 
but do elicit a greater frequency of report, 
ynder t circumstances, increases in inter- 
. vievyer speech duration affect the content of the 
^ respondent speech without affecting its dura^ 
tion. Somehow a long^question provides cues 
which tend to elicit more information from the 
respondent even though the response duration 
stays practically unchanged. Tentative explana^ 
tions of this question length effect will be 
proposed in the conclusions to this section, 
along with an experimental design for the test of 
some derived hypotheses. " 

V EXPERIMENT 2: EFFECTS OF 
QUESTION LENGTH ON VALIDITY 
OF REPORT 

While there is some evidence from experiment 

■ 1 that increases in question length result in 
greater reporting frequency, no data are avail- 
able so^ far on the validity of: the extra infprma= 

vtion obtained by long questions. It may be that ^ 
respondents overreported a number of health 
events or conditions, as is known to occur 
sometimes in health interviews, Howeverj there 
is no particular hypothesis to predict that 
Qverreporting; would increase as a function of 
question length/ On the contrary, drawing again 
from the cue-search model o f theiinterview, it is 
hypothesized that respondents may interpret 

■ long questions as calling for completeness and 
accuracy of report. This would decrease under- 
reporting, and possibly dverreporting, while im- 
proving the overall validity of the data, 

:?"^The purpose of experiment 2^ was to ascer- 
tain the validity of health information reported 
in response to long questions. To achieve this 
^objective,^ a sample- of respondents was dra^^ 
from a population of patients who had visited a 
physician in a prepaid clinic during a 6-month . 
period prior to the survey. At the time of the 

checklist, of IS chronic conditions indicating 
whether the patient had or did not have each 

^^a^For a ftill report of this experiment, which aliolnduded an 

M the effteti of verbs! rcinforcimem and reinter- 1: 
view, ice reference 53*^ v ; , :- - 



listed condition, or whether sufficient diagnosti^^^^ 
information was available. Information about 
the patient was obtained by4he physician from' 
the patient's record and from lis own knowl* 
edge of the patient's healthV A weighted sample 
of persons was used in v^hich 88 percent had at^ 
least one of the listed chronic conditions and 12 
percent had none of them. Respondents were 
white^males, 18 to 60 years of age who resided 
. . in the greater Detroit metropolitan area. 

Among other experimental techniques used m 
the study was a test of the effects of question 
length on accuracy of reporting, A questionnaire 
was prepared using standard short questions. 
These call for the reporting of various health 
conditions and behaviors. In the middle of the 
questii^naire, checklist-type questions were ^ 
troduced which asked about the presence or 
^sence ofLthe ^lS chronic conditi^^ 
the physician siimmary form* ; ■ " 

A second version of the questibrinaire was 
prepared using the same questions but in long 
form. Questions were lengthened ^ in a way 
somewhat comparable to the method used in 
experiment 1 . Since experiment 1 had shown a 
. carryover effect of question length from long to 
shorty questions, it was hypothesized ■thkt corti'^ ^^^^ 
trast between questions of various lengths rather 
than specific question length per se was the 
operating variable. In order to increase this 
contrast effect, a mixture of questions S^yi^^ 
length, rather than the previous large block^^^ 
short and long questions, was used. Also, in 
contrast to the procedure used in the first 
experiment, standard short probe questions vyere 
introduced after all items in both questionnaire 
forms. This meant that the more information a 
respondent reported^ the more short probe 
qu^^stions she would be asked. This procedure 
was aimed at decreasing the contrast in length 
between the two questionnaire forms as a 
function of reporting frequency, thus minimiz» j 
ing the expected effect of question length on the 
number of items reported. Finally,^ the st rateg>^ # 
used in lengthening the questions was slightly 

ments as in the original experiment, while pthers^ ^; 
contained only two. 

Thus, while question length was implemented 
on the. basis of v comparable principles i (redun- j 
JMicy, cliches, aMEfillersl^ith^the 
characteristics of the question held consta • 
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less differentiation was probably attained be- 
tween the two experimental treatments. This 
might result in some minimization of the ques- 
tion length effect, 

: V Ten white female interviewers were employed 
in the study. Qu^ition treatments, long and 
shorty were assigrieH at _ random within geo- 
graphic clusters of respondents with each jnter- 
^dewe^ administering both types of interviews. 
Budget .considerations led to some compromise, 
both in terms of experimental design and sample 
size, which might be reflected in less than 
optimal stability of the results. One hundred and 
six persons were interviewed with the short- 
question procedure and 96 with the long- 
question procedure. 

Since, for various reasons, errors can exist in 
the physician forms as well as in the. respondent 
reports, a high , degree of agreement between the 
two sources was not "expected. However, it was 
assumed that better validity of the respondent 
reporting would increase the agreement rates 
between the two sources. In other words^ if 
agreement rates on presence and absence of 
chronic conditions were found to be : higher 
under long-question interviews than under short- 
question intemews, presum^ly an improve- 
ment in validity had been obtained in the former 
procedure. 

Probability of agreement between physician 
and respondent on the presence or absence of 
the listed 13 chronic conditions was computed 
on the basis of match and mismatch , in *^ Yes" r 
and "No" responses provided by the two infor-- 
mation sources. Excluding c^es where, the 
physician or respondent lacked sufficient infor- 
mation to determine the presence of a condi- 
tion, the following four possibilities of match or 
mismatch existed for each chronic condition: 
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Result and Discussion 

The original report of the study treated the 
data in terms of two types of mismatch: : 

Type X = ^ and type Y ^ gfig- 

Type X mismatch was expected to represent the. 
extent of underreporting (false negative re- 
sponses) r type Y, the extent of overreporting 
(false positive responses), assuming that the 
physician was right in his evaluation. According 
to the analysis, the overreporting error was 
relatively infrequent and was not affected by the: 
question length (.10 under both interview tech- 
niques). On the other hand, the underreporting 
error was more frequent and it was reduced 
significantly (p < .05) in the interyicws in which- 
long questions were used (.46 with short ques- 
tions and .38 with long questions). 

Since one may reasonably question the ade- 
quacy of the physician- s report^ as well as the 
validity of respondent report^ an analysis of the 
data from a slightly different point of view is: 
presented. For both interview treatments, table 
44 shows probabilities of agreement relating to 
the pr^jence of chronic conditions. 

Table 45 presents the corresponding probabil- 
ities of agreement on the adi^nc^ of chronic 
conditions. _ _ ^ - ' . . 

This new approach avoids the assumption that 
the medical: records . were more valid than the 
refpondent*s report. It does assume that greater 
agreement between the two sources indicates 
higher validity of the reported data. 

Based on the material shown in table 44, 
whether one starts froni the physician data (row 
1), or firom the respondent data (row 2), or from i 
both (row 3) j the probability of agreement on r 
the existence of a chronic condition is con- 
iiitently higher with interviews using long ques- ^ 
tions than it is with interviews using short-; 
questions. For two agreement rates the obtained ' 
improvement is statistically significant. The in- 
crease in overall probability of agreement ob- 
tained with long-questiqn interviews amounts to 
16 percent. . ~ 

The data in table 45 show that while agree- 
ment rates on the absence of chronic conditions 
are not noticeably enhanced by the use of long>: 
questions, neither are they diminished by' ques- 
tion length. ' r i 



= TabI& 44, Probabni'iy of physieian^rtspondtnt agretmer* en th# 



presence of chronis eenditlQni, by typt of agreemint rati and 
qutstion length (original inten/iew) 



Type of egreemtnt rates : 


: Questionnaire 
procedure^ 




Percent 
Increase 
due to 

length 


Short 
questions 


Long 
questions 


Difference 


A 

■ A+B 


Probability that chronic conditions ehacked m present by the 
physician were rtported as preientby rBspondent ..... 


.537 


.622 


^.086 


+17 


A 

A+C 


Frobebiiity that ghronig conditions reported as present by the 
respondent were cheeked as present by the physician . . . . . . . . ; 


A77 
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+1 


A 

A+i+C 


Overall probability of agreenient between physician and respondent 
for chronic conditionrfifi&ntloned ii pressnt by either of them .... 


.338 


.392 


3.054 





^Number of persons inten/iewed were 106 In short^question procedure ind^ii 



< .05, one-tailed, based 6h^. 

< JO, one^tailed, based on Z. 



6 in long^uestion procedure. 



Table 45. Probability of physidan-respondent agreement on the absence of chronic conditions, by type of agfeement rate and 

question length (original interview) 



Type of agreement rates 



^ Probability that chronic conditiont checked m absent by the 
D+C physician were reported as absent by the respondent . . 

P_ Probability that chronic conditions reported as absent by the 

D+B - respondent were checked as absent by the physician . . . 



: — y — Overall probability of agrtennent between physician and respondent f- 
D+e+C for chronic conditions mentioned as absent by either of tham ; 



Questionnaire 
" procedure^ 



Short 
questions 



.901 



.921 



.836 



J'!*'H'^b?r of persons interviewed were 106 In ihort^uestlon proctdure and 96 In 



Long \.\ 
questions 



.901 



.934 



.847 



Qifference 



long-question procedure. 



.013 



,011 



Percent 
increase 

due to 
question : 

length : 



+1 



+1 



.^^ T com^ 
plement to the earlier findings. Exp^^ 
has shown that 'more information is elicited by 
long questions than by short questions, and the 

^ present experiment indicates that long^uei 
also elicit information, of higher validity. 

: Even though the study was designed primarily 
to test the validity of reporting of chronic ^ 

i^condi tipns^^ rather^ than ^simply ^ the.;, amount ;ire- ^ 

fported,vattention w^ also given to the numbe 
of .reported conditions. From the list of 13 
chronic conditions, ■ an average of 2,26 were 

^accounted -for inT the long-question treatment as ^ 

icpinpared to 2.06 in the. short-question treat- 
ment. Although this amounts to only a 10- 



percent increase in . the number of chronic 
^^nditions reported; it ■ is ^riot riegli^ble in view 
of the fact that the recall task is easier and the 
likelihood ; of - responding is expected to be 
uniformlyv higher when recognition stimuli like 
checklist questions are provided.^ ^ For health 
■ be^ayior noted: in other parts o f the question- 
naire, the effect /of tquestion length on number 
—PNtems^reported-was^'modest- and not s 

*^rily significant. Several reasons for a minimiza- 
tion of the expected effects of question length 
in this study were proposed earlier* 

Since reinterview was another variable to be 
investigated in <the study, 50 percent of the 
respondents selected for original interview were 



designated to be contacted 2 weeks later.^ The 
content of the followup was very close to that 
of the original interview. Some new questions 
about health insurance ware introduced to avoid 
excessive ? repetition. The questionnaire con- 
tained the identical chronic conditions checklist* 
vAJl respondents who originally had been given 
short-question interviews were given short- 
question reinterviews; respondents originally 
asked long ques tio ns were rein tervie wed with 



, ^For re^ons of field efficiency^ all feipondenti oripnally 
intcn^cwcd durinf the first half of the data collection period 
were designated for reinterview* 



long questions* Thus, data were available to 
examine potential variations in agreement rates 
from the first interview to the later one. While 
agreement rates were poorer the second time 
than they were originally for both short- and 
long-question treatmentSj the deterioration in 
validity was lower with long questions than it 
was with short questions. " 

Since one focus of this study is question 
length rather than reinterview, tables 46 and 47 
show the same type of agreement rates oris 
presence and absence of chronic conditions as 
presented in tables 44 and 45, but computed 
this time on the basis of the reinterview data- 



Table 48, Probability of phystcian-respondint ^reement on the presence of chronic conditions, by type of agreement rate and 

question length (reinterview} 



Tyi^ of agreement rates : 


Questionnaire 
procedure^ 


Difference 


Percent . 
insrease ^ 

due to 
question . 

length 


Short 
questions 


Long 
questions . 


--^^A^^ Prebabtliiy that chrDnic conditions checked as prBBent by the 

A+B >^ phyiician were reported as present by the respondent . i . -.■ * , . * 


.B33 


.iS7 


.064 


+12 


_jA Probability that chronic conditions reported ai present by the 
^ A+C respondent were checked as present by the physician . * ^ . , * , . . 


MB 


M27 




+2i 


A Overall probability of agreement between physictan and resoondent 
A+B+G for €hronic conditions mentioned as present by either of them . . * . 


^ ioi 


,38e 


^.088 


+29 



Number of persons reinters'iewed were 49 in short<;uestian procedure end 63 in long^uestion procedure. 
^ .05, one-tailed, based on 



Table 47* Probability of physician^respondent agreement on the absence of chronic conditions, by type of agreement rate and 





question length (reinten/iew) 










Type of agreement rates , 


Questionnaire 
- procedure' 


Difference 


Percent 
increase 

due to 
qyestion 

length = 


Short 
qu^tions 


Long 
questions 


D 
D+C 


Probability that chronic conditions checked at absent by the 
physician were reported as absent by the respondent 




- .194 




+3 


D 
D+B 


Probability that chronic conditions reported as absent by the 
respondent checked as absent by the physician 1 . , , » ..... . 




.919 


.003 




D 

D+B+e 


Overall probability of agreement between physician and respondent 
: for chronic conditions mentioned as absent by either of thenn . . , , 


.804 


.829 


.025 


+3 ' 



r ; ^Number of persons reinterviewed were 49 in short-question procedure and §3 in long<|uestion procedure. 
; < ,10, one-tailed, based on Z. 

^More than zero, but less than 1 percent. ■ ^ . 
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; Air three agreement ratei shown in table 46 
r . were substantially improved by the use of long 
questions in reinterview. The obtained increaie 
: in two of the agreement rates reaches statistical 
. significance (p < ,05). The increase in overall 
probability of agreement between physician and 
respondent on the presence of chronic condi- 
: tions amounts to 29 percent. 

While increases in agreement on the absence 
of conditions due tO: question length are very 
modest (table 47), there is indication again that 
no- deterioration occurred for this type of 
agreement as a result of question length in 
reinterviews. For the first agreement rate, the 
increase obtained in long»question reinten^iews 
reached statistical significance at the 10-percent 
leyeL Thus, the data ob tain^'d the second time 
confirm the findirigs^Qbtained originally. Long 
questions elicit a more valid report than do short 
questions under conditions of reinterviewj as 
well as under conditions of initial interview. 
In summary, this second experiment, demon- 
; strated that improvement in the validity of the 
reporting of Health conditions can be achieved 
without changing the content or meaning of the 
questions, but only by increasing their length* 
Somehow the retrieval of more accurate infor- 
mation is facilitated by a question of long 
duration. The; request for accurate reporting is 
implicitly conveyed by long questions, even 
5 though the explicit demand for information and 
apparen tly the response duration are unchanged, 

: CONCLUSIONS 



On the basis of the data prrsanted, three 
major suggestions have been proposed which can 
be summarized as foUows: When the length of 
survey interview questions is substantially in- 
creased and their information dempnd,.,h^^ 
; constant (a) no appreciable incre^e is obtained { 
in V response duration; yet^ (b) the response 
contains more information; and (c) the reported 
information is more valid. - 

The firs t sugge s tio n co n tradic t s 6 1 her research 
^inT^hi dh ri t^asT found that there ^ was a speech^^^ 
;length ipatching effect in the interview. -This 
: matching effect in other research might have 
resulted from ' an uncontrolled increase in the 
iriformatibn demanded by long questions* The 
present study indicates that, when information 



demand is controlledi long questions do not 
elicit long responses: EKperimental manipula- 
tions of information demanded, associated with 
a control imposed on question lengthy might 
help to solve the issue in further research. 

That long questions in comparison with short 
ones might elicit more information and a more 
accurate report is contradictory to co^ "»n 
assumptions and current sun^ey metho iugy. 
However, the evidence in this paper leads to the 
conclusion that lengthy and redundant ques- 
tions, as designed in this study^ elicit increased 
accuracy of reports even though the responses 
do not last any longer than those in response to 
short questions. These findings are puzzling, and 
they raise questions of importance to survey 
research. The following should be considered as 
results of preliminary investigations that require 
replication^ . 

a* The length of the question has cueing 
effects upon reporting behavior, causing in- 
creased accuracy but not extending to response 
duration* A long question may convey to the 
respondent : the idea that^ ^becaiise the inters 
viewer has spent much time asking the question^ 
the task is important md, therefore, requires 
serious efforts* Furthermore, a long question 
may indicate to the respondent that the inter- 
viewer is not in a hurry, and thus releases 
perception-of-time constraints possibly detri- 
mental to adequate performance in regular 
interviews* Finally, the responding behavior may 
also gain in ; effectiveness because some) of the 
initial ruminating=type activity has already taken 
place while the question was being asked, A long 
question may therefore provide cues leading to 
more adequate performance and at the same 
time prepare the respondent for the expression 
of more efficient verb^ behaviorT 

b. Question length or interviewer speech 
duration is only a vehicle or a proxy for another 
influential variable^ namely, the time given for 
recall - activity, A long question increases the 
time available for search activity and thus 
improves the outcome of recall, 
V Sin cfe Tincreases' iri^ question " length have 
been implemmted partially by introducing re- 
dundancy in question wording, the multiple 
presentation of the stimulus may be the2influ- 
ential variable* It may act either because of 
increased exposure time or because of a repeated 
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trials effect*? Finally » it may be that redundancy 
improves the clarity of the question which also 
leads to better reporting performance. 

According to this analysis^ the effects of at 
least three variables should be investigated in 
' further research: question length per se and its 
cue-giving properties; time provided by the 
question for recall activity; and redundancy of 
the : question, ; In the experiments described 
earlier in this paper, all three dimensions have 
been varied simultaneously so that the specific 
effect of each could not be isolated. In the 
experimental treatments, questions were longer | 
they also provided more time for recall since 
their first statement always referred to a major 
part of the question content; and they were 
redundant* 

The following design proposal is an attempt 
to investigate the specific effects of variations in 
total question length and recall time, while 
partially controlling for redundancy. Experi- 
■ mental questions could be designed using the 
following pool of statements— equal in length- 
according to various arrangements: 

Q - question in its short standard form; 
F ^ filler statement inlToducing extraneous 
information of inconsequential character 
and unrelated to the specific question 
demand; and 
q = introductory statement describing the 
topic of the question in a manner suffi- 
cient to stimulate relevant recall activity. 
Each experimental questionnaire would con- 
tain only one type of question structures as 
described below: 

Questionnaire 1= Q: questions in their short 
standard form* Question length and recall 
time are low- 

Questionnaire 2 ^ FQ: the short question pre- 
ceded by a filler statement. The total length 
of the question is roughly doubled, whereas 
the time for recall activity is unchanged since 
; the filler is entirely unrelated to the question 

__demand. Question length is medium and recall 
time is low. 

^estionnaire 3 ^ gQ^ the short question pre- 
ceded this time by a statement introducing 

: the major question demand. Question length ^ 
and recall time are both medium. 

8 



Qiestionnaire 4 - FqQ: the short question is 
preceded by the introductory statement^ 
which is itself preceded by an irrelevant fillers- 
question length is high, and recall time is 
medium. ' 

Questionnaire 5 = gFQ: the short question is 
preceded by a filler, which is itself preceded 
by an introductory statement. Question 
length and recall time are both high. 

Redundancy occurs whenever a question uses 
both q and Q, so that this variable is controlled 
within groups of treatments 1-2 whero there is 
no redundancy and within groups where 
redundancy has been introduced* 

This experimental design is presented in figure 
2, where each cell represents one questiorinalfe 
procedure and indicates the strategy used in^^ 
question wording for this procedure. Compari- 
son of data from cells 1 and 2 will detect the 
effect of increased length under conditions of 
equal recall time and no redundancy. Compari- 
son of cells 3 and 4 wilf^detect an effect of 
increased length with equal recall time and 
redundancy/ A comparison of cells 2 and 3 will 
explore a combined effect of increased recall 
time and redundancy, under conditions of equal 
length. Comparison of cells 4 and 5 will detect 
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the effect of increased recall times with equal 
length and equal redundancy. Diagonal compari» 
mm of cells will provide a means of examining 
the effects of varioui combinations of the three 
variables^ The application of this design may 
generate three types of outcomes: 



a* A replication of the earlier findings 5 

b. An identification of the most efficient 
strategy or question anatomy; and 

c. Some identification of the variables causing 
improvement in reporting. 
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APPENDIX 



APPLICATION OFSURVEY RESEARCH CENTER 
FINDINGSTO HEALTH INTERVIEW 
SURVEY PROCEDURES 



Based on the experience of many researchers 
in the health field and other areas it was 
expected that underreporting would be one of 
the major problems to be solved in the data 
collection phase of the Health Interview Survey. 
However, before any attempt could be made to 
remedy this shortcoming of the interview 
method, it was necessary to obtain some idea of 
the magnitude of the problem, to learn some- 
thing about the characteristics of personi who 
fail to report health evehtSs and to identify the 
particular kinds of events that tend to be 
underreported. The need for this kind of infor- 
mation led to a number of studies involving a 
comparison of information provided by inter- 
view respondents with independent medical 
records of known validity. Several of these 
studies; conducted by the Survey Research 
Genter (SRC) of the University of Michigan, 
provided information that led to major revisions 
in the collection and processing of Health 
Interview Survey (HIS) datap 

Recall of Heallii iven^ 

One of the early studies carried out by SRC in 
1958-59 consisted of a comparison of hospitali- 
zations repbrted in interviews with actual hos- 
pital records. A sample of patients discharged 
from hospitals participating=in-the Professional 
Activity Study were fntcn^iewedj and the results 
were compared with discharge records (see 
ref, 17 for a complete description of this study). 
Findifigs of this study, described earlier in this 
report, demonstrate that underreporting of hos- 
pitaiization in a health interview situation is 
' irifluericedlsy the impact of the hbspitalization. 



the threat or embarrassment caused by the 
nature of the condition causing hospitalization, 
and the time elapsed between the interview and 
the period spent in the hospitaL Since the first 
two of these findings involved intrinsic charac- 
teristics of the health event, no immediate 
solution to underreporting associated with these 
causes was available* 

However, the consistent increase in under- 
reporting with the time elapsed between hos- 
pitalization and the interview was a finding that 
appeared to have practical applicalion to data 
collected in the Health Interview Survey. As a 
result, in the derivation of estimates of the 
volurhe of hospital discharges from the basic HIS 
data collected, beginning in July 1958, the 
6-month period preceding the date of interview: 
was used as the period of reference. By doubling 
the weight attached to each of the reported 
events within that period, it was possible to 
produce estimates comparable to those based on 
12 months of recall, but with considerably less 
of the underreporting bias introduced by the use 
of the longer recall period. 

The use of the IZ-month recall period was 
continued in the collection of data on hospital 
experience because of the several kinds of 
estimates produced from the Survey. To com- 
bine the hospital episodes of sample persons in. 
order to estimate the number of persons with 
one or more episodes in a given year, it is 
necessary to consider a year's experience for 
each sample person. On the othe:r hand, in 
estimating the annual volume of hospital dis- 
charges, any recall period can be used if the 
weight attached to each event in the estimating^ 
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procedure irproperly adjusted. Since the length 
of the recall period is inverse! related to the 
magnitude of the sampling errur, the 6^month 
reference period was selected so that response 
bias could be appreciably reduced without an 
undue increase in the size of the sampling error. 

The imprecision with which respondents re^ 
called dates of health events during an interview, 
brought to light by several of the SRC studies! 
led to the use of a recall period in the collection 
phase of hospital data extending beyond the 
year preceding the interview. For example, 
persons interviewed during July of a particular 
year were asked about their hospital experience 
since May of the previous year. This innovation 
improved the reporting of events that occurred 
near the beginning of the reference year, as well 
as hospitalizations that started prior to the year 
preceding the interview but extending into the 
reference year. During the processing phase, 
those hospitaHzations for which no days during 
the year prior to interview were recorded were 
eliminated from the hospitalization data. 

The SRC record-check studies also revealed 
inaccuracies in the reporting of physician visits 
Even though the recall period for the reporting 
of physician visits was limited to the 2-week 
period prior to week of interview, some of the 
visits occurring during that period were not 
reported and, in other instances, visits occurring 
prior to the period were reported as happening 
within the recall period. This finding eventually 
led to the decision to enumerate physician visits 
on the questionnaire by date of visit so that 
comparison of the occurrence and interview 
dates would establish that the event had oc- 
curred during the appropriate recall period. 

Effective Probing for Health Evan^ 

; ^ Several of the Survey Research Center studies 
have indicated that much of the information not 
reported during an interview has not been 
^!P^^^4u^?I Jj5S.it disappeared, from memory. 
It was simply not elicited because the question- 
mg procedures failed to bring it forth. This 

finding suggested that the probe questions de- 
signed oto. encourage the reporting of health 
events in the HIS were not stimulating retrieval 
□f information sufficiently; and that questions 
constructed to elicit certain kinds of response 
should be added* 



Prior to July 1962, respondents in the HIS 
were asked about overnight stays in hospitals of 
family members during the previous 12 months, 
and about stays in nursing homes and sani' 
tariums. Comparison of the estimates of hos= 
pitalizations for delivery derived from these data 
with natality figures for the years 1968^62 
indicated that hospitalizations of this type were 
underreported in the interview survey. To cor- 
rect^ this situation, a probe question directed 
particularly to the population at risk was added 
to the questionnaire. In households where chil- 
dren 1 year of age or under were reported as 
household members, the following probe ques- 
uons were asked: "When was (the child) bom?" ; 
"Was (the Jiild) born in a hospital?" "Is this 
hospitalization included in the number you gave 
mep'Mf the hospitalization had occurred during 
the reference period and it had not been 
reported in response to eariier probes, then the 
entries on the questionnaire for the mother and 
the child were corrected. The addition of this 
series of questions resulted in an appreciable 
decrease in the amount of underreporting in this 
area of the questionnaire. 

Through June f964, the reporting of infbrma^ 
tion on the number of physician visits during the 
2 weeks prior to week of Interview was depend- 
ent on one probe question: "Last week or the 
week before, did anyone in the family talk to a 
doctor or go to a doctor's office^ or clinic^" 
Bepnning in January 196^ the next period 
during which information on physician visits was ' 
collected in the Survey, two probe questions 
were added: /'During that S^week period has 
anyone in the family been to a doctor's office or 
. chnic ^for shots,.X^rays, tests, or examinati^s?" 
and, "During that period, did anyone in the 
family get any medical advice from a doctor 
over the telephone?" The first of these questions 
was added to remind the respondent of visits 
that vi/ere made for preventive care or, in some 
instances, .for reasons other-than treatment of 
illness. The second question informed the 
respondent that telephone calls to obtain medi- . 
cal advice were consic -red as physician vists in ■ 
the Survey, Both of these questions had been 
used by SRC in the study designed to evaluate 
interviewer performance over time. 

Because these probe questions were added at " 
the same time as the question>egarding the date ^ ^ 
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of the viiit, their effect on the data was not as 
obvious as it otherwise would have been* The 
procedure of relating the date of the occurrence 
of a visit to the date of interview to determine if 
it actually occurred during the proper recall 
period effectively excluded all overreporting of 
visits that had actually occurred prior to the 
recall period or during the interview week. 
Previously such visits v^ould have compensated 
for some of the underreporting. Their removal 
from the data made it difficult to evaluate the 
effectiveness of the added probe questions in 
terms of additional visits reportcds but there is 
evidence that the yield from these questions was 
substantial* 

I ntarviewi r- R esponden t C om m u n iaation 

Many of the studies conducted by SRC have 
emphasized the importance of the influence the 
interviewer exerts on the respondent and, in 
turn, on the completeness and accuracy of the 
reported data* The interviewer's attitude, her 
expectationSp the kind of feedback she provides, 
and her behavior during the interview are only a 
few of the factors that determine the kind and 
anlount of information obtained during an 
interview. However, to take advantage of this 
phenomenon in order to improve the quality of 
reported information, controls must be initiated 
to avoid the introduction of interviewer biases. 
One of the best methods of exereising control of 
interviewer behavior is to include devices, quesv 
tions, and statements in the questionnaire which 
will improve communication between the 
participants in the interview but will not direct 
the responses.' 

Some of the innovations in the HIS question- 
naire that have resulted from this type of 
research include the following: 

a, A simple introductory statement has been 
prepared in which the interviewer identifies 
.herself_at_the ,dopn^of_the^_hpu 
explains very briefly the purpose of her 
visit. In case the respondent (or another 
family member) wants to know more about 
: the purpose of the survey or the uses of the 
collected data^ a more detailed statement is 
available to the interviewer, 
b* Within the questionnaire, introductory 
statements are used to expilain the subject 



matter about which questions are to be 
asked and to serve as transition devices 
from one health topic to another* For 
example, a section on X-iay visits was 
introduced by the statement^ **Exposure to 
all kinds of X-rays Is a matter of particular 
interest to the Public Health Services and I 
have some questions about X-rays and 
fluoroscopes.'' 

c. A small calendar cards with the appropriate 
2-week recall period outlined in red is 

. handed to the respondent early in the 
interview so that she is constantly aware of 
the specific 2-week period referred to 
throughout the interview. 

d. Nondirective probe questions have been 
included on the questionnaires in areas 
where nonspecific or ambiguous informa- 
tion is. likely to be reported. For example, 
if the respondent reports that she visited 
the doctor at a clinic, the interviewer is 
instructed to ask: "Was it a hospital out- 
patient clinic, a compMiy clinic^ or some 
other kind of clinic?"' 

In the SRC study on interviewer performance 
over time, it was found that interviewers be- 
came less careful or conscientious in using the 
techniques they were trained to use • There was 
also e^dence that interviewers who performed 
well inspired their respondents to perform well, 
as measured by the reporting of hospitalizations. 
This study also brought to light the need for 
interviewer training to include devices for stimu- 
lating the interviewer's enthusiasm for her job in 
addition to retraining in the use of interviewing 
techniques* These findings have been reinforced 
by records of interviewer performance main- 
tained by the Bureau of the Census, and have 
been taken into account in the preparation of 
material for the periodic training and retraining 
sessions conducted for interviewers in the Health 
Interview Survey. 

Otfier Coniiderationi 

As in any series of research studies, some of 
the experimental measures in the SRC series, 
when tested as methods for reducing .under- 
reporting in interviews, did not contribute any;^ 
significant findings.Tn other instances, encourag- 
ing results from studies were either inconclusiye 
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or needed further testing in a nonlaboratory 
situation. In this latter category is the finding 
that long questions are more effective than short 
ones in bringing forth c mplete and accurate 
responses. More research is needed to determine 
if long questions are productive because they 
have cuing effects on reporting behavior j allow 
more time for recall activity, or merely because 
they introduce redundancy of stimuli. Until the 
specific variables causing improved reporting are 
identified, the introduction of longer questions 
on the HIS questionnaire, which would lead to 
longer interviews and increased costSj could not 
be justified. 

Verbal reinforcement by the interviewer has 
been shown to have cognitive and motivational 



effects on the respond^'nt by instilling awareness 
of respondent task requirements and encourage 
ing adequate responses to subsequent questions. 
However, it will be necessary to develop ways in 
which the interviewer can effectively use re- 
inforcement without introducing an undue 
amount of bias in the collected data before this 
device can be seriously considered. 

All of the studies in which the SRC has 
attempted to measure the amounts and types of 
underrepoirting in inter\dews indicate that some 
basic principles of memory and retrieval can be 
used to improve reporting. Further research is 
needed on the ways in which information is 
stored and on effective methods of retrieving 
that information. 
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VITAL AND HEALTH STATISTICS PUBLICATIONS SERIES 



, Formerly Pitblic HmUh Service Publication No, 1000 

Series 1. Programs and Collection Procedures.-^RepQm which describe the general programs of the National 
: Center for Health Statistics and its offices and divisions, data coUection methods used, definitions, and 
^ other material nicessary for imderstanding the data. 

Series 2. Data Evaluation and Methods Hesearch^Sludieu of new statistical methodology including experimental 
tests of new survey methods, studies of vital statistics collection methods, new analytical techniques, 
- ^bjectiveevaluationsof reliability of collected data, com 

Series 3, Analytical Studies, -^Repons pTts^nimg or interprethfe studies based on vital and health 

statistics, carrying the analysis further thaai the expository types of reports in the other series* 

S^ies 4. Documents and Com7nittce Reports, ^¥'m^l reports of major committees concerned with vital and 
health statistics, and documents such as recommended model vital registration laws and revised birth 
and death certificates. ■• . 

Series 10. Data from the Health Interview Survey, SuiMcB on illness* accidental injuries; disability s use of 
hospital, medical, dental, and other services; and other health-related topics, based on data collected in 
a continuing national household interview survey. \- - . , ^ 

Series 11, Data from the Health Examination Survey.=DatB. from direct examination, testing, and measurement 
of riational samples of the civilian, nonins tit utionalized population provide the basis for two types of 
reports! (I) estimates of the medically defined prevalence of specific diseases in the United States and 
the distributions of the population with respect to physical, physiological, and psychological cKarac- 
teristics; and (2) analysis of relationships among the various measurements without reference to an 
explicit finite unive^c of persons. 

Series 12, Data from the InsHtutionalned Population Siin^gys.-^Discontinued effective 1975. Future reports from 
these surveys will be in Series 13. . 

Series 13. Data on Health Resources Utilization,— BtMBticB on the utilisation of health manpower and facilities 
providing long-term Care, ambulatory care, hospital care, and family planning services. - 

Series 14, Data 071 Health Resources: Manpower and Facilities, -^Smhiim on the numbers, geographic distrib- 
ution, and characteristics of health resources including physicians, dentists, nurses, other health occu= 
pations, hospitals, nursing homesj and outpatient faciliries* : 'f : , . 

Series 20. Data on Mortality.— VsLnous staristics on mortality other than as included in regular annual or monthly 
repons* Special analyses by cause of death, age, and other demographic vaiiablcji; geographic and time 
series analyses I and statistics on characteristics of deaths not available from the vital records, based on 
sample surveys of those records; ; ^ 

Series 21, Data on Natality, Marriage, and Divorce,— Vmous statistics on natality, marriage, and divorce other 
than as included in regular annual o^ monthly reports. Speciar analyses by demographic variables; 
geographic and time series analyses i studies of fertility; and staristics on characteristics of birtlis not 
available from the vital recordi^-based on sample sim?eys of those recordsr — — ^«_™^^ 

Series 22, Data from the National Mortality and Natality Siiri^^j.-^Discontinued effective 1975. Future reports 
: ; , from these sample surveys based on vital records mil be included in Series 20 and 2i; respectively. 

Series 23. DaUx from the National Survey of Family Growth. -^Smhtics on fertility, family formation and diss© - 
; . lution, family ptaning, and related maternal and infant health topics derived from a biennial survey of 

. r a nationwide probability sample of ever-married women 15-44 years of age*. 

For a'list of titles of reports published in these fcries, write loi Scientific and. Technical Information Branch 
: v : , . . * National Gcntcr for Hcalth Statlstics 

. . . : :\. V"/ Public Health Sem^ . ; 

' i — . . . , RockvillcMd. 20857 : 



